[==========] Running 38 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 37 tests from ExternalMiniClusterTest
[ RUN ] ExternalMiniClusterTest.TestKerberosReacquire
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:08 dist-test-slave-bw88 krb5kdc[26779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:08 dist-test-slave-bw88 krb5kdc[26779](info): set up 2 sockets
Aug 09 07:48:08 dist-test-slave-bw88 krb5kdc[26779](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 07:48:11.027202 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.291s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:48:11 dist-test-slave-bw88 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:11Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:48:11.278744 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:46063
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:46063 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:11.663920 26804 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:11.664386 26804 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:11.664592 26804 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:11.672909 26804 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:11.673043 26804 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:11.673170 26804 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:11.673252 26804 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:11.684500 26804 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:46063
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:46063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:11.686710 26804 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:11.688772 26804 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:11.707717 26812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:11.707687 26810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:11.709599 26809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:11.710135 26804 server_base.cc:1061] running on GCE node
I20260809 07:48:11.711369 26804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:11.713750 26804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:11.715023 26804 hybrid_clock.cc:648] HybridClock initialized: now 1786261691714940 us; error 104 us; skew 500 ppm
I20260809 07:48:11.715569 26804 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:48:11 dist-test-slave-bw88 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261691, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:11.719295 26804 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:48:11.721853 26804 webserver.cc:533] Webserver started at http://127.26.37.62:37289/ using document root <none> and password file <none>
I20260809 07:48:11.722898 26804 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:11.723067 26804 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:11.723598 26804 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:11.727054 26804 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "e2e40a1264c44a6482ee654be2b8e4de"
format_stamp: "Formatted at 2026-08-09 07:48:11 on dist-test-slave-bw88"
I20260809 07:48:11.728242 26804 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "e2e40a1264c44a6482ee654be2b8e4de"
format_stamp: "Formatted at 2026-08-09 07:48:11 on dist-test-slave-bw88"
I20260809 07:48:11.735911 26804 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260809 07:48:11.741576 26822 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:11.744043 26804 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260809 07:48:11.744365 26804 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "e2e40a1264c44a6482ee654be2b8e4de"
format_stamp: "Formatted at 2026-08-09 07:48:11 on dist-test-slave-bw88"
I20260809 07:48:11.744596 26804 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:11.762933 26804 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:11.767765 26804 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:11.768165 26804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:11.798552 26804 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:46063
I20260809 07:48:11.798588 26879 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:46063 every 8 connection(s)
I20260809 07:48:11.801224 26804 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:11.809803 26880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:11.810431 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26804
I20260809 07:48:11.810797 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:48:11.828157 26880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Bootstrap starting.
I20260809 07:48:11.834574 26880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:11.836277 26880 log.cc:826] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:11.840098 26880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: No bootstrap required, opened a new log
Aug 09 07:48:11 dist-test-slave-bw88 krb5kdc[26779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:48:11.849231 26880 raft_consensus.cc:359] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } }
I20260809 07:48:11.849994 26880 raft_consensus.cc:385] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:11.850230 26880 raft_consensus.cc:740] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e40a1264c44a6482ee654be2b8e4de, State: Initialized, Role: FOLLOWER
I20260809 07:48:11.851747 26880 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } }
I20260809 07:48:11.852185 26880 raft_consensus.cc:399] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:11.852504 26880 raft_consensus.cc:493] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:11.852799 26880 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:11.855516 26880 raft_consensus.cc:515] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } }
I20260809 07:48:11.856678 26880 leader_election.cc:304] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2e40a1264c44a6482ee654be2b8e4de; no voters:
I20260809 07:48:11.857826 26880 leader_election.cc:290] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:11.858116 26887 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:11.859536 26887 raft_consensus.cc:697] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 1 LEADER]: Becoming Leader. State: Replica: e2e40a1264c44a6482ee654be2b8e4de, State: Running, Role: LEADER
I20260809 07:48:11.860399 26887 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } }
I20260809 07:48:11.860921 26880 sys_catalog.cc:565] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:11.866724 26888 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2e40a1264c44a6482ee654be2b8e4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } } }
I20260809 07:48:11.866756 26889 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2e40a1264c44a6482ee654be2b8e4de. Latest consensus state: current_term: 1 leader_uuid: "e2e40a1264c44a6482ee654be2b8e4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } } }
I20260809 07:48:11.867264 26889 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:11.867269 26888 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:11.869210 26898 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:11.878490 26898 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:11.894169 26898 catalog_manager.cc:1383] Generated new cluster ID: ab06f7ae22c44e4991c9e892b69682fb
I20260809 07:48:11.894366 26898 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:11.916743 26898 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:48:11.918121 26898 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:11.934158 26898 catalog_manager.cc:6092] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Generated new TSK 0
I20260809 07:48:11.935262 26898 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:48:12.038811 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:46063
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:12.387444 26917 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:12.387761 26917 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:12.387852 26917 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:12.395963 26917 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:12.396171 26917 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:12.406499 26917 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:46063
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:12.408095 26917 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:12.409483 26917 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:12.419477 26925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:12.419602 26924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:12.421607 26917 server_base.cc:1061] running on GCE node
W20260809 07:48:12.422331 26927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:12.423002 26917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:12.423898 26917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:12.425091 26917 hybrid_clock.cc:648] HybridClock initialized: now 1786261692425050 us; error 36 us; skew 500 ppm
I20260809 07:48:12.425475 26917 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:48:12 dist-test-slave-bw88 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261692, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:12.428638 26917 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:48:12.430219 26917 webserver.cc:533] Webserver started at http://127.26.37.1:45187/ using document root <none> and password file <none>
I20260809 07:48:12.430894 26917 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:12.431046 26917 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:12.431331 26917 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:12.433864 26917 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "04008eb9b05744849202aebd7d5b4fc5"
format_stamp: "Formatted at 2026-08-09 07:48:12 on dist-test-slave-bw88"
I20260809 07:48:12.434690 26917 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "04008eb9b05744849202aebd7d5b4fc5"
format_stamp: "Formatted at 2026-08-09 07:48:12 on dist-test-slave-bw88"
I20260809 07:48:12.440224 26917 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:48:12.443600 26935 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:12.444861 26917 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:12.445039 26917 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "04008eb9b05744849202aebd7d5b4fc5"
format_stamp: "Formatted at 2026-08-09 07:48:12 on dist-test-slave-bw88"
I20260809 07:48:12.445216 26917 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:12.470588 26917 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:12.474817 26917 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:12.475119 26917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:12.476228 26917 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:12.478211 26917 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:12.478348 26917 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:12.478499 26917 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:12.478606 26917 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:12.516278 26917 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34639
I20260809 07:48:12.516326 27091 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34639 every 8 connection(s)
I20260809 07:48:12.517922 26917 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:12.526820 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26917
I20260809 07:48:12.527071 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:48:12 dist-test-slave-bw88 krb5kdc[26779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261692, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:48:12.539016 27092 heartbeater.cc:344] Connected to a master server at 127.26.37.62:46063
I20260809 07:48:12.539522 27092 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:12.540496 27092 heartbeater.cc:507] Master 127.26.37.62:46063 requested a full tablet report, sending...
I20260809 07:48:12.544322 26842 ts_manager.cc:194] Registered new tserver with Master: 04008eb9b05744849202aebd7d5b4fc5 (127.26.37.1:34639)
I20260809 07:48:12.546023 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:48:12.546741 26842 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:54009
I20260809 07:48:13.550200 27092 heartbeater.cc:499] Master 127.26.37.62:46063 was elected leader, sending a full tablet report...
Aug 09 07:48:26 dist-test-slave-bw88 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261706, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:26.721688 26816 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 07:48:27 dist-test-slave-bw88 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261707, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:27.430958 26930 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 07:48:28 dist-test-slave-bw88 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260809 07:48:28.580215 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26804
I20260809 07:48:28.582273 26875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:28.696241 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26804
I20260809 07:48:28.729592 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:46063
--webserver_interface=127.26.37.62
--webserver_port=37289
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:46063 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:29.078514 27101 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:29.078887 27101 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:29.079056 27101 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:29.089218 27101 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:29.089352 27101 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:29.089447 27101 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:29.089532 27101 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:29.101701 27101 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:46063
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:46063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=37289
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:29.103308 27101 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:29.104645 27101 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:29.113397 27112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:29.113621 27110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:29.113412 27109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:29.114177 27101 server_base.cc:1061] running on GCE node
I20260809 07:48:29.114997 27101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:29.116426 27101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:29.117718 27101 hybrid_clock.cc:648] HybridClock initialized: now 1786261709117654 us; error 55 us; skew 500 ppm
I20260809 07:48:29.118103 27101 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:48:29 dist-test-slave-bw88 krb5kdc[26779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:29.121409 27101 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:48:29.123071 27101 webserver.cc:533] Webserver started at http://127.26.37.62:37289/ using document root <none> and password file <none>
I20260809 07:48:29.123821 27101 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:29.123975 27101 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:29.130002 27101 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:48:29.133009 27123 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:29.134428 27101 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:29.134632 27101 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "e2e40a1264c44a6482ee654be2b8e4de"
format_stamp: "Formatted at 2026-08-09 07:48:11 on dist-test-slave-bw88"
I20260809 07:48:29.135385 27101 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:29.155491 27101 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:29.162601 27101 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:29.162896 27101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:29.183910 27101 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:46063
I20260809 07:48:29.183944 27196 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:46063 every 8 connection(s)
I20260809 07:48:29.185657 27101 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:29.191836 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27101
I20260809 07:48:29.193679 27198 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:29.197512 27198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Bootstrap starting.
Aug 09 07:48:29 dist-test-slave-bw88 krb5kdc[26779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:48:29.218921 27198 log.cc:826] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:29.232121 27198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:29.232789 27198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Bootstrap complete.
I20260809 07:48:29.238826 27198 raft_consensus.cc:359] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } }
I20260809 07:48:29.239420 27198 raft_consensus.cc:740] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e40a1264c44a6482ee654be2b8e4de, State: Initialized, Role: FOLLOWER
I20260809 07:48:29.240069 27198 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } }
I20260809 07:48:29.240381 27198 raft_consensus.cc:399] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:29.240526 27198 raft_consensus.cc:493] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:29.240788 27198 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:29.244776 27198 raft_consensus.cc:515] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } }
I20260809 07:48:29.245313 27198 leader_election.cc:304] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2e40a1264c44a6482ee654be2b8e4de; no voters:
I20260809 07:48:29.245980 27198 leader_election.cc:290] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 07:48:29.246276 27204 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:48:29.246985 27204 raft_consensus.cc:697] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [term 2 LEADER]: Becoming Leader. State: Replica: e2e40a1264c44a6482ee654be2b8e4de, State: Running, Role: LEADER
I20260809 07:48:29.247550 27204 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } }
I20260809 07:48:29.248051 27198 sys_catalog.cc:565] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:29.250203 27205 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e2e40a1264c44a6482ee654be2b8e4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } } }
I20260809 07:48:29.250551 27205 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:29.250602 27206 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2e40a1264c44a6482ee654be2b8e4de. Latest consensus state: current_term: 2 leader_uuid: "e2e40a1264c44a6482ee654be2b8e4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e40a1264c44a6482ee654be2b8e4de" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 46063 } } }
I20260809 07:48:29.251530 27206 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:29.252388 27211 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:29.258672 27211 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:29.260176 27211 catalog_manager.cc:1295] Loaded cluster ID: ab06f7ae22c44e4991c9e892b69682fb
I20260809 07:48:29.260306 27211 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:29.263173 27211 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:29.264364 27211 catalog_manager.cc:6103] T 00000000000000000000000000000000 P e2e40a1264c44a6482ee654be2b8e4de: Loaded TSK: 0
I20260809 07:48:29.265911 27211 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:29.707713 27147 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "04008eb9b05744849202aebd7d5b4fc5" instance_seqno: 1786261692507433) as {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:50109; Asking this server to re-register.
I20260809 07:48:29.708694 27092 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:29.709067 27092 heartbeater.cc:507] Master 127.26.37.62:46063 requested a full tablet report, sending...
I20260809 07:48:29.710724 27147 ts_manager.cc:194] Registered new tserver with Master: 04008eb9b05744849202aebd7d5b4fc5 (127.26.37.1:34639)
I20260809 07:48:29.720760 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:48:29.721016 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26917
Aug 09 07:48:29 dist-test-slave-bw88 krb5kdc[26779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:48:29.742910 27087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:29.832167 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26917
I20260809 07:48:29.856076 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27101
I20260809 07:48:29.857429 27192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:29.962148 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27101
2026-08-09T07:48:29Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestKerberosReacquire (21293 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/0
I20260809 07:48:30.015609 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36365
--webserver_interface=127.26.37.62
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351 with env {}
W20260809 07:48:30.367568 27232 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:30.367887 27232 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:30.368080 27232 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:30.376266 27232 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:30.376359 27232 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:30.376456 27232 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:30.376554 27232 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:30.386530 27232 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:30.388233 27232 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:30.390161 27232 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:30.398419 27241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:30.398422 27239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:30.399646 27243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:30.400050 27232 server_base.cc:1061] running on GCE node
I20260809 07:48:30.400787 27232 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:30.400975 27232 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:30.401103 27232 hybrid_clock.cc:648] HybridClock initialized: now 1786261710401076 us; error 0 us; skew 500 ppm
I20260809 07:48:30.401522 27232 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:30.404305 27232 webserver.cc:533] Webserver started at http://127.26.37.62:46519/ using document root <none> and password file <none>
I20260809 07:48:30.405109 27232 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:30.405233 27232 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:30.405673 27232 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:30.408100 27232 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "58dfa892910444ac91b8096dccc8b227"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:30.408928 27232 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "58dfa892910444ac91b8096dccc8b227"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:30.414450 27232 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:48:30.417871 27255 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:30.419230 27232 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:48:30.419425 27232 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "58dfa892910444ac91b8096dccc8b227"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:30.419646 27232 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:30.436224 27232 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:30.437233 27232 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:30.437570 27232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:30.459581 27232 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36365
I20260809 07:48:30.459604 27324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36365 every 8 connection(s)
I20260809 07:48:30.461305 27232 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:30.465866 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27232
I20260809 07:48:30.466200 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:48:30.466531 27325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:30.467958 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38631
--webserver_interface=127.26.37.61
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351 with env {}
I20260809 07:48:30.476204 27325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:30.495031 27325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:30.496096 27259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38631: connect: Connection refused (error 111)
W20260809 07:48:30.497879 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38631: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38631: connect: Connection refused (error 111)
I20260809 07:48:30.547395 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } attempt: 1
W20260809 07:48:30.549793 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38631: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38631: connect: Connection refused (error 111)
I20260809 07:48:30.618146 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } attempt: 2
W20260809 07:48:30.620397 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38631: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38631: connect: Connection refused (error 111)
I20260809 07:48:30.711758 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } attempt: 3
W20260809 07:48:30.715204 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38631: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38631: connect: Connection refused (error 111)
W20260809 07:48:30.828987 27327 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:30.829540 27327 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:30.829703 27327 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:30.838352 27327 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:30.838511 27327 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:30.838626 27327 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:30.838717 27327 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:30.849195 27327 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:30.851109 27327 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:30.853165 27327 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:48:30.858592 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } attempt: 4
W20260809 07:48:30.862128 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38631: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38631: connect: Connection refused (error 111)
W20260809 07:48:30.862166 27337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:30.862434 27338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:30.863363 27342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:30.864835 27327 server_base.cc:1061] running on GCE node
I20260809 07:48:30.865752 27327 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:30.865906 27327 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:30.865978 27327 hybrid_clock.cc:648] HybridClock initialized: now 1786261710865956 us; error 0 us; skew 500 ppm
I20260809 07:48:30.866254 27327 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:30.868999 27327 webserver.cc:533] Webserver started at http://127.26.37.61:39283/ using document root <none> and password file <none>
I20260809 07:48:30.869824 27327 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:30.869927 27327 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:30.870416 27327 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:30.872781 27327 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "5832ac12571c44e5ae371595359e27d9"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:30.873639 27327 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "5832ac12571c44e5ae371595359e27d9"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:30.879060 27327 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:48:30.882496 27349 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:30.883891 27327 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:30.884037 27327 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "5832ac12571c44e5ae371595359e27d9"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:30.884208 27327 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:30.904829 27327 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:30.905773 27327 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:30.906021 27327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:30.927286 27327 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:38631
I20260809 07:48:30.927328 27423 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:38631 every 8 connection(s)
I20260809 07:48:30.928875 27327 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:48:30.931622 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27327
I20260809 07:48:30.931842 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:48:30.933285 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:40351
--webserver_interface=127.26.37.60
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351 with env {}
I20260809 07:48:30.934595 27424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:30.943012 27424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:30.957580 27424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:30.967412 27424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:30.968837 27355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
W20260809 07:48:30.970415 27424 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
I20260809 07:48:31.019781 27424 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 1
W20260809 07:48:31.021986 27424 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
I20260809 07:48:31.090205 27424 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 2
W20260809 07:48:31.092471 27424 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
I20260809 07:48:31.161494 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } attempt: 5
I20260809 07:48:31.169262 27325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:31.171293 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
I20260809 07:48:31.183703 27424 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 3
W20260809 07:48:31.185963 27424 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
I20260809 07:48:31.222565 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 1
W20260809 07:48:31.224750 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
I20260809 07:48:31.293015 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 2
W20260809 07:48:31.295156 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
W20260809 07:48:31.297060 27428 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:31.297438 27428 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:31.297531 27428 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:31.305876 27428 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:31.305961 27428 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:31.306020 27428 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:31.306068 27428 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:31.315434 27428 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:40351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:31.317085 27428 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:31.318933 27428 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:31.327265 27441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:31.327343 27440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:31.328949 27445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:31.329200 27424 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 4
I20260809 07:48:31.329491 27428 server_base.cc:1061] running on GCE node
I20260809 07:48:31.330477 27428 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:31.330724 27428 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:31.330852 27428 hybrid_clock.cc:648] HybridClock initialized: now 1786261711330823 us; error 0 us; skew 500 ppm
I20260809 07:48:31.331337 27428 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260809 07:48:31.331493 27424 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
I20260809 07:48:31.334200 27428 webserver.cc:533] Webserver started at http://127.26.37.60:43771/ using document root <none> and password file <none>
I20260809 07:48:31.334971 27428 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:31.335130 27428 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:31.335515 27428 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:31.337922 27428 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "3752a28e549149d5bc655d0a712edcbe"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-bw88"
I20260809 07:48:31.338704 27428 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "3752a28e549149d5bc655d0a712edcbe"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-bw88"
I20260809 07:48:31.343750 27428 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:48:31.347090 27452 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:31.348340 27428 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:48:31.348522 27428 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "3752a28e549149d5bc655d0a712edcbe"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-bw88"
I20260809 07:48:31.348735 27428 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:31.381901 27428 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:31.382858 27428 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:31.383157 27428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:31.401401 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 3
W20260809 07:48:31.403590 27325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111)
I20260809 07:48:31.404489 27428 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:40351
I20260809 07:48:31.404528 27520 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:40351 every 8 connection(s)
I20260809 07:48:31.406373 27428 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:48:31.410822 27521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:31.416424 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27428
I20260809 07:48:31.416622 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:48:31.416836 27521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:31.437788 27521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:31.443930 27521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:31.454591 27521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Bootstrap starting.
I20260809 07:48:31.458093 27521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:31.459261 27521 log.cc:826] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:31.462005 27521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: No bootstrap required, opened a new log
I20260809 07:48:31.467459 27521 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.467770 27521 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:31.467907 27521 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3752a28e549149d5bc655d0a712edcbe, State: Initialized, Role: FOLLOWER
I20260809 07:48:31.468488 27521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.469409 27531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.469704 27531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:31.470443 27521 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:48:31.480379 27545 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:31.480520 27545 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:31.478382 27428 master_runner.cc:187] Error getting master registration for 127.26.37.62:36365: OK, instance_id { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" instance_seqno: 1786261710453618 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:48:31.485440 27428 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:31.580842 27325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 4
I20260809 07:48:31.592774 27325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap starting.
I20260809 07:48:31.597118 27325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:31.598630 27325 log.cc:826] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:31.602247 27325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: No bootstrap required, opened a new log
I20260809 07:48:31.607422 27325 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.607710 27325 raft_consensus.cc:385] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:31.607849 27325 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58dfa892910444ac91b8096dccc8b227, State: Initialized, Role: FOLLOWER
I20260809 07:48:31.608424 27325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.609434 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.609745 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:31.610620 27325 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:48:31.621717 27562 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:31.621851 27562 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:31.623519 27232 master_runner.cc:187] Error getting master registration for 127.26.37.61:38631: OK, instance_id { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" instance_seqno: 1786261710921661 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:48:31.630131 27232 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:31.630726 27424 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } attempt: 5
W20260809 07:48:31.635243 27460 tablet.cc:2406] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:31.644387 27424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: Bootstrap starting.
I20260809 07:48:31.648906 27424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:31.650401 27424 log.cc:826] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:31.654047 27424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: No bootstrap required, opened a new log
I20260809 07:48:31.662127 27424 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.662484 27424 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:31.662622 27424 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5832ac12571c44e5ae371595359e27d9, State: Initialized, Role: FOLLOWER
I20260809 07:48:31.663367 27424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.665091 27531 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:31.665417 27531 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.667742 27531 leader_election.cc:290] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58dfa892910444ac91b8096dccc8b227 (127.26.37.62:36365), 5832ac12571c44e5ae371595359e27d9 (127.26.37.61:38631)
I20260809 07:48:31.669013 27291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3752a28e549149d5bc655d0a712edcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58dfa892910444ac91b8096dccc8b227" is_pre_election: true
I20260809 07:48:31.669785 27291 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3752a28e549149d5bc655d0a712edcbe in term 0.
I20260809 07:48:31.671113 27456 leader_election.cc:304] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3752a28e549149d5bc655d0a712edcbe, 58dfa892910444ac91b8096dccc8b227; no voters:
I20260809 07:48:31.672049 27531 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:31.672212 27531 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:31.672672 27531 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:31.673655 27566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.673982 27566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:31.675897 27424 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:31.675966 27391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3752a28e549149d5bc655d0a712edcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5832ac12571c44e5ae371595359e27d9" is_pre_election: true
I20260809 07:48:31.676479 27391 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3752a28e549149d5bc655d0a712edcbe in term 0.
I20260809 07:48:31.678980 27531 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.680052 27531 leader_election.cc:290] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [CANDIDATE]: Term 1 election: Requested vote from peers 58dfa892910444ac91b8096dccc8b227 (127.26.37.62:36365), 5832ac12571c44e5ae371595359e27d9 (127.26.37.61:38631)
I20260809 07:48:31.681576 27391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3752a28e549149d5bc655d0a712edcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5832ac12571c44e5ae371595359e27d9"
I20260809 07:48:31.681793 27391 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:31.682855 27291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3752a28e549149d5bc655d0a712edcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58dfa892910444ac91b8096dccc8b227"
I20260809 07:48:31.683108 27291 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:31.690533 27291 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3752a28e549149d5bc655d0a712edcbe in term 1.
W20260809 07:48:31.691201 27262 tablet.cc:2406] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:31.694729 27456 leader_election.cc:304] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3752a28e549149d5bc655d0a712edcbe, 58dfa892910444ac91b8096dccc8b227; no voters:
I20260809 07:48:31.695817 27391 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3752a28e549149d5bc655d0a712edcbe in term 1.
I20260809 07:48:31.696641 27531 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:31.709774 27531 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 1 LEADER]: Becoming Leader. State: Replica: 3752a28e549149d5bc655d0a712edcbe, State: Running, Role: LEADER
I20260809 07:48:31.710551 27531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:31.726792 27568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3752a28e549149d5bc655d0a712edcbe. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.727283 27568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:31.728310 27578 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:31.730988 27578 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:31.748673 27291 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Refusing update from remote peer 3752a28e549149d5bc655d0a712edcbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:31.752945 27391 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 1 FOLLOWER]: Refusing update from remote peer 3752a28e549149d5bc655d0a712edcbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:31.754474 27585 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:48:31.754758 27568 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:31.755713 27568 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:31.780297 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3752a28e549149d5bc655d0a712edcbe. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.780596 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:31.785449 27581 mvcc.cc:204] Tried to move back new op lower bound from 7316527971305111552 to 7316527971175940096. Current Snapshot: MvccSnapshot[applied={T|T < 7316527971305111552}]
I20260809 07:48:31.788295 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.788537 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:31.794828 27566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3752a28e549149d5bc655d0a712edcbe. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.795120 27566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:31.796334 27587 mvcc.cc:204] Tried to move back new op lower bound from 7316527971305111552 to 7316527971175940096. Current Snapshot: MvccSnapshot[applied={T|T < 7316527971305111552}]
I20260809 07:48:31.797235 27578 catalog_manager.cc:1383] Generated new cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:31.797399 27578 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:31.799201 27568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.799434 27568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:31.799768 27568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.799961 27568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:31.802414 27566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:31.802584 27566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:31.828553 27578 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:48:31.830642 27578 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:31.847482 27578 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Generated new TSK 0
I20260809 07:48:31.848970 27578 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:31.859807 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--time_source=system_unsync with env {}
W20260809 07:48:32.230844 27594 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:32.231213 27594 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:32.231362 27594 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:32.239318 27594 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:32.239528 27594 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:32.248989 27594 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:32.250682 27594 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:32.252578 27594 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:32.262629 27606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:32.262655 27605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:32.263803 27608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:32.264194 27594 server_base.cc:1061] running on GCE node
I20260809 07:48:32.264880 27594 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:32.265091 27594 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:32.265235 27594 hybrid_clock.cc:648] HybridClock initialized: now 1786261712265191 us; error 0 us; skew 500 ppm
I20260809 07:48:32.265605 27594 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:32.268036 27594 webserver.cc:533] Webserver started at http://127.26.37.1:42171/ using document root <none> and password file <none>
I20260809 07:48:32.268858 27594 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:32.269013 27594 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:32.269495 27594 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:32.272293 27594 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:32.273173 27594 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:32.279192 27594 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 07:48:32.282908 27613 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:32.284348 27594 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:48:32.284555 27594 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:32.284795 27594 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:32.313437 27594 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:32.314563 27594 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:32.314955 27594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:32.316042 27594 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:32.317952 27594 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:32.318084 27594 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:32.318241 27594 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:32.318349 27594 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:32.357568 27594 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46625
I20260809 07:48:32.357589 27755 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46625 every 8 connection(s)
I20260809 07:48:32.359182 27594 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:32.361913 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27594
I20260809 07:48:32.362169 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:32.370255 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--time_source=system_unsync with env {}
I20260809 07:48:32.386061 27756 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:32.386540 27756 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:32.387776 27756 heartbeater.cc:507] Master 127.26.37.60:40351 requested a full tablet report, sending...
I20260809 07:48:32.389328 27758 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38631
I20260809 07:48:32.389515 27758 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:32.390077 27758 heartbeater.cc:507] Master 127.26.37.61:38631 requested a full tablet report, sending...
I20260809 07:48:32.390630 27473 ts_manager.cc:194] Registered new tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:46625)
I20260809 07:48:32.390995 27757 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:32.391146 27757 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:32.391698 27757 heartbeater.cc:507] Master 127.26.37.62:36365 requested a full tablet report, sending...
I20260809 07:48:32.392052 27376 ts_manager.cc:194] Registered new tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:46625)
I20260809 07:48:32.392736 27473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:50759
I20260809 07:48:32.394222 27280 ts_manager.cc:194] Registered new tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:46625)
I20260809 07:48:32.624615 27562 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:32.624761 27562 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: loading cluster ID for follower catalog manager: success
I20260809 07:48:32.628844 27562 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: acquiring CA information for follower catalog manager: success
I20260809 07:48:32.631518 27562 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:48:32.743636 27769 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:32.743940 27769 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:32.744019 27769 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:32.752345 27769 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:32.752504 27769 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:48:32.760017 27585 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:32.760169 27585 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: loading cluster ID for follower catalog manager: success
I20260809 07:48:32.762871 27769 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:32.764374 27585 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: acquiring CA information for follower catalog manager: success
I20260809 07:48:32.764510 27769 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:32.766392 27769 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:48:32.766885 27585 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:48:32.776079 27778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:32.777238 27777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:32.777957 27780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:32.779433 27769 server_base.cc:1061] running on GCE node
I20260809 07:48:32.780088 27769 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:32.780267 27769 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:32.780390 27769 hybrid_clock.cc:648] HybridClock initialized: now 1786261712780352 us; error 0 us; skew 500 ppm
I20260809 07:48:32.780720 27769 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:32.783214 27769 webserver.cc:533] Webserver started at http://127.26.37.2:40809/ using document root <none> and password file <none>
I20260809 07:48:32.783973 27769 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:32.784103 27769 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:32.784466 27769 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:32.786950 27769 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "9aff321a559b4c6ba39d521c2a3eef98"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:32.787757 27769 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "9aff321a559b4c6ba39d521c2a3eef98"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:32.793543 27769 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 07:48:32.797034 27787 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:32.798488 27769 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:32.798696 27769 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "9aff321a559b4c6ba39d521c2a3eef98"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:32.798936 27769 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:32.831960 27769 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:32.832988 27769 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:32.833387 27769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:32.834692 27769 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:32.836822 27769 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:32.836949 27769 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:32.837095 27769 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:32.837200 27769 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:32.890479 27769 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:34769
I20260809 07:48:32.890539 27929 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:34769 every 8 connection(s)
I20260809 07:48:32.892043 27769 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:32.900674 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27769
I20260809 07:48:32.900882 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:32.907127 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--time_source=system_unsync with env {}
I20260809 07:48:32.944375 27933 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:32.944767 27933 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:32.946123 27933 heartbeater.cc:507] Master 127.26.37.60:40351 requested a full tablet report, sending...
I20260809 07:48:32.947880 27934 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:32.948028 27934 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:32.948230 27473 ts_manager.cc:194] Registered new tserver with Master: 9aff321a559b4c6ba39d521c2a3eef98 (127.26.37.2:34769)
I20260809 07:48:32.948536 27934 heartbeater.cc:507] Master 127.26.37.62:36365 requested a full tablet report, sending...
I20260809 07:48:32.949361 27473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:44661
I20260809 07:48:32.949724 27280 ts_manager.cc:194] Registered new tserver with Master: 9aff321a559b4c6ba39d521c2a3eef98 (127.26.37.2:34769)
I20260809 07:48:32.962023 27935 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38631
I20260809 07:48:32.962198 27935 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:32.962608 27935 heartbeater.cc:507] Master 127.26.37.61:38631 requested a full tablet report, sending...
I20260809 07:48:32.963925 27376 ts_manager.cc:194] Registered new tserver with Master: 9aff321a559b4c6ba39d521c2a3eef98 (127.26.37.2:34769)
W20260809 07:48:33.303680 27945 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:33.304080 27945 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:33.304215 27945 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:33.312811 27945 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:33.313022 27945 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:48:33.323082 27945 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:33.324770 27945 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:33.326757 27945 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:33.337050 27952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:33.337380 27957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:33.337086 27953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:33.338902 27945 server_base.cc:1061] running on GCE node
I20260809 07:48:33.339485 27945 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:33.339689 27945 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:33.339782 27945 hybrid_clock.cc:648] HybridClock initialized: now 1786261713339760 us; error 0 us; skew 500 ppm
I20260809 07:48:33.340068 27945 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:33.342367 27945 webserver.cc:533] Webserver started at http://127.26.37.3:38505/ using document root <none> and password file <none>
I20260809 07:48:33.343150 27945 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:33.343302 27945 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:33.343688 27945 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:33.346150 27945 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "ee2286de73d64942935bc447e02088d3"
format_stamp: "Formatted at 2026-08-09 07:48:33 on dist-test-slave-bw88"
I20260809 07:48:33.346974 27945 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "ee2286de73d64942935bc447e02088d3"
format_stamp: "Formatted at 2026-08-09 07:48:33 on dist-test-slave-bw88"
I20260809 07:48:33.352391 27945 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:48:33.356043 27963 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:33.357501 27945 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:48:33.357692 27945 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "ee2286de73d64942935bc447e02088d3"
format_stamp: "Formatted at 2026-08-09 07:48:33 on dist-test-slave-bw88"
I20260809 07:48:33.357910 27945 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:33.377657 27945 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:33.378584 27945 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:33.378885 27945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:33.379891 27945 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:33.381767 27945 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:33.381907 27945 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:33.382046 27945 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:33.382136 27945 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:33.395466 27756 heartbeater.cc:499] Master 127.26.37.60:40351 was elected leader, sending a full tablet report...
I20260809 07:48:33.422876 27945 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:39351
I20260809 07:48:33.422924 28104 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:39351 every 8 connection(s)
I20260809 07:48:33.424388 27945 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:33.431622 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27945
I20260809 07:48:33.431831 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:33.442903 28106 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:33.442942 28108 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38631
I20260809 07:48:33.443387 28108 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:33.443390 28106 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:33.443818 28105 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:33.443977 28105 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:33.444590 28106 heartbeater.cc:507] Master 127.26.37.62:36365 requested a full tablet report, sending...
I20260809 07:48:33.444705 28108 heartbeater.cc:507] Master 127.26.37.61:38631 requested a full tablet report, sending...
I20260809 07:48:33.444728 28105 heartbeater.cc:507] Master 127.26.37.60:40351 requested a full tablet report, sending...
I20260809 07:48:33.446506 27473 ts_manager.cc:194] Registered new tserver with Master: ee2286de73d64942935bc447e02088d3 (127.26.37.3:39351)
I20260809 07:48:33.446692 27280 ts_manager.cc:194] Registered new tserver with Master: ee2286de73d64942935bc447e02088d3 (127.26.37.3:39351)
I20260809 07:48:33.447357 27376 ts_manager.cc:194] Registered new tserver with Master: ee2286de73d64942935bc447e02088d3 (127.26.37.3:39351)
I20260809 07:48:33.447573 27473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:54107
I20260809 07:48:33.457564 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:48:33.465989 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:48:33.466248 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27232
I20260809 07:48:33.467228 27319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:33.574700 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27232
I20260809 07:48:33.605305 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36365
--webserver_interface=127.26.37.62
--webserver_port=46519
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351 with env {}
W20260809 07:48:33.874490 27456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:36365: connect: Connection refused (error 111)
W20260809 07:48:33.876307 27456 consensus_peers.cc:597] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe -> Peer 58dfa892910444ac91b8096dccc8b227 (127.26.37.62:36365): Couldn't send request to peer 58dfa892910444ac91b8096dccc8b227. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:36365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:48:33.951952 27933 heartbeater.cc:499] Master 127.26.37.60:40351 was elected leader, sending a full tablet report...
W20260809 07:48:33.952718 27934 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:36365 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:36365: connect: Connection refused (error 111)
W20260809 07:48:33.957875 28114 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:33.958230 28114 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:33.958361 28114 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:33.967288 28114 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:33.967411 28114 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:33.967530 28114 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:33.967615 28114 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:33.977813 28114 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=46519
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:33.979513 28114 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:33.981436 28114 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:33.989653 28127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:33.990162 28129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:33.990339 28126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:33.991125 28114 server_base.cc:1061] running on GCE node
I20260809 07:48:33.991886 28114 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:33.992082 28114 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:33.992175 28114 hybrid_clock.cc:648] HybridClock initialized: now 1786261713992137 us; error 0 us; skew 500 ppm
I20260809 07:48:33.992513 28114 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:33.995211 28114 webserver.cc:533] Webserver started at http://127.26.37.62:46519/ using document root <none> and password file <none>
I20260809 07:48:33.995994 28114 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:33.996143 28114 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:34.001711 28114 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:48:34.004587 28134 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:34.006059 28114 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:34.006245 28114 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "58dfa892910444ac91b8096dccc8b227"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:34.007015 28114 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:34.039490 28114 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:34.040463 28114 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:34.040773 28114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:34.061910 28192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36365 every 8 connection(s)
I20260809 07:48:34.061923 28114 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36365
I20260809 07:48:34.063870 28114 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:34.065790 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28114
I20260809 07:48:34.066182 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27594
I20260809 07:48:34.072899 28193 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:34.076541 27750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:34.077607 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap starting.
I20260809 07:48:34.101893 28193 log.cc:826] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:34.112768 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:34.113394 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap complete.
I20260809 07:48:34.123241 28193 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:34.124193 28193 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58dfa892910444ac91b8096dccc8b227, State: Initialized, Role: FOLLOWER
I20260809 07:48:34.125082 28193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:34.126040 28201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:34.126343 28201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:34.127292 28193 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:34.138304 28219 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:34.138530 28219 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: loading cluster ID for follower catalog manager: success
I20260809 07:48:34.143893 28219 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: acquiring CA information for follower catalog manager: success
I20260809 07:48:34.147070 28219 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:34.205614 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27594
I20260809 07:48:34.230531 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46625
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--webserver_port=42171
--webserver_interface=127.26.37.1
--time_source=system_unsync with env {}
I20260809 07:48:34.450136 28105 heartbeater.cc:499] Master 127.26.37.60:40351 was elected leader, sending a full tablet report...
I20260809 07:48:34.460410 28153 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ee2286de73d64942935bc447e02088d3" instance_seqno: 1786261713414060) as {username='slave'} at 127.26.37.3:59597; Asking this server to re-register.
I20260809 07:48:34.461472 28106 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:34.463716 28153 ts_manager.cc:194] Registered new tserver with Master: ee2286de73d64942935bc447e02088d3 (127.26.37.3:39351)
I20260809 07:48:34.486460 27568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:34.486723 27568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:34.520990 28201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3752a28e549149d5bc655d0a712edcbe. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:34.521394 28201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:48:34.603943 28228 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:34.604379 28228 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:34.604480 28228 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:34.612839 28228 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:34.612998 28228 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:34.623435 28228 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46625
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42171
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:34.625018 28228 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:34.626933 28228 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:34.636858 28242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:34.636898 28243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:34.637843 28246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:34.638401 28228 server_base.cc:1061] running on GCE node
I20260809 07:48:34.638978 28228 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:34.639178 28228 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:34.639277 28228 hybrid_clock.cc:648] HybridClock initialized: now 1786261714639254 us; error 0 us; skew 500 ppm
I20260809 07:48:34.639599 28228 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:34.641944 28228 webserver.cc:533] Webserver started at http://127.26.37.1:42171/ using document root <none> and password file <none>
I20260809 07:48:34.642746 28228 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:34.642903 28228 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:34.648293 28228 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 07:48:34.651088 28253 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:34.652400 28228 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:48:34.652612 28228 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:34.653421 28228 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:34.682665 28228 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:34.683588 28228 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:34.683889 28228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:34.684895 28228 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:34.686758 28228 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:34.686862 28228 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:34.687016 28228 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:34.687090 28228 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:34.723723 28228 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46625
I20260809 07:48:34.723770 28407 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46625 every 8 connection(s)
I20260809 07:48:34.725395 28228 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:34.733446 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28228
I20260809 07:48:34.734018 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28228
I20260809 07:48:34.756939 28409 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:34.757036 28410 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38631
I20260809 07:48:34.757470 28409 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:34.757479 28410 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:34.757915 28408 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:34.758085 28408 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:34.758972 28408 heartbeater.cc:507] Master 127.26.37.60:40351 requested a full tablet report, sending...
I20260809 07:48:34.759075 28409 heartbeater.cc:507] Master 127.26.37.62:36365 requested a full tablet report, sending...
I20260809 07:48:34.759269 28401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:34.846360 28410 heartbeater.cc:507] Master 127.26.37.61:38631 requested a full tablet report, sending...
I20260809 07:48:34.847911 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28228
I20260809 07:48:34.848498 27473 ts_manager.cc:194] Re-registered known tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:46625)
I20260809 07:48:34.848578 28153 ts_manager.cc:194] Registered new tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:46625)
I20260809 07:48:34.849632 27473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:33437
I20260809 07:48:34.854692 27376 ts_manager.cc:194] Re-registered known tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:46625)
W20260809 07:48:34.873150 27454 connection.cc:573] server connection from 127.26.37.1:33437 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:48:34.873145 28135 connection.cc:573] server connection from 127.26.37.1:55043 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:48:34.873050 27354 connection.cc:573] server connection from 127.26.37.1:37901 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:48:34.874415 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46625
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--webserver_port=42171
--webserver_interface=127.26.37.1
--time_source=system_unsync
--fault_before_start=1.0 with env {}
I20260809 07:48:34.960728 27934 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:34.961880 28153 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "9aff321a559b4c6ba39d521c2a3eef98" instance_seqno: 1786261712879570) as {username='slave'} at 127.26.37.2:36857; Asking this server to re-register.
I20260809 07:48:34.962438 27934 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:34.963603 28153 ts_manager.cc:194] Registered new tserver with Master: 9aff321a559b4c6ba39d521c2a3eef98 (127.26.37.2:34769)
I20260809 07:48:34.994884 28231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3752a28e549149d5bc655d0a712edcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:34.995226 28231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: LEADER
W20260809 07:48:35.234076 28421 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:35.234370 28421 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:35.234512 28421 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:48:35.234601 28421 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:35.242592 28421 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:35.242832 28421 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:35.252488 28421 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46625
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42171
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:35.254554 28421 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:35.256510 28421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:35.266428 28428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:35.266426 28429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:35.267701 28432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:35.267943 28421 server_base.cc:1061] running on GCE node
I20260809 07:48:35.268551 28421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:35.268716 28421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:35.268838 28421 hybrid_clock.cc:648] HybridClock initialized: now 1786261715268800 us; error 0 us; skew 500 ppm
I20260809 07:48:35.269225 28421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:35.271517 28421 webserver.cc:533] Webserver started at http://127.26.37.1:42171/ using document root <none> and password file <none>
I20260809 07:48:35.272228 28421 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:35.272339 28421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:35.277786 28421 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:48:35.280313 28438 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.281683 28421 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:35.281854 28421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:35.282589 28421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:35.313060 28421 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:35.314039 28421 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:35.314338 28421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:35.315402 28421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:35.317131 28421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:35.317262 28421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.317420 28421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:35.317520 28421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:48:35.317922 28421 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:48:35.318045 28421 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:48:35.318138 28421 fault_injection.cc:59] injecting fault for kudu (pid 28421): FLAGS_fault_before_start (process will exit)
I20260809 07:48:35.345989 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46625
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--webserver_port=42171
--webserver_interface=127.26.37.1
--time_source=system_unsync with env {}
W20260809 07:48:35.694941 28447 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:35.695253 28447 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:35.695341 28447 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:35.703562 28447 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:35.703727 28447 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:35.713637 28447 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46625
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42171
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:35.715261 28447 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:35.717010 28447 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:35.727233 28454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:35.727275 28455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:35.728020 28457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:35.729080 28447 server_base.cc:1061] running on GCE node
I20260809 07:48:35.729764 28447 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:35.729961 28447 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:35.730077 28447 hybrid_clock.cc:648] HybridClock initialized: now 1786261715730052 us; error 0 us; skew 500 ppm
I20260809 07:48:35.730460 28447 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:35.732712 28447 webserver.cc:533] Webserver started at http://127.26.37.1:42171/ using document root <none> and password file <none>
I20260809 07:48:35.733734 28447 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:35.733879 28447 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:35.740254 28447 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:48:35.742998 28462 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.744369 28447 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:48:35.744570 28447 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:35.745396 28447 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:35.764508 28447 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:35.765496 28447 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:35.765801 28447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:35.766728 28447 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:35.768617 28447 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:35.768698 28447 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.768812 28447 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:35.768860 28447 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.803910 28447 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46625
I20260809 07:48:35.803956 28616 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46625 every 8 connection(s)
I20260809 07:48:35.805621 28447 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:35.807468 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28447
I20260809 07:48:35.807943 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28447
I20260809 07:48:35.838339 28609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:35.923672 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28447
I20260809 07:48:35.943985 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27769
W20260809 07:48:35.944985 28632 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:48:35.834096 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:37051 (local address 127.26.37.62:36365)
0809 07:48:35.834685 (+ 589us) server_negotiation.cc:207] Beginning negotiation
0809 07:48:35.834694 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:48:35.837408 (+ 2714us) server_negotiation.cc:408] Connection header received
0809 07:48:35.837610 (+ 202us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:48:35.837616 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:48:35.837698 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:48:35.837824 (+ 126us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:48:35.923436 (+ 85612us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.924842 (+ 1406us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.943770 (+ 18928us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:37051: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":310,"threads_started":1}
W20260809 07:48:35.945865 28628 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:48:35.810815 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:50557 (local address 127.26.37.61:38631)
0809 07:48:35.837514 (+ 26699us) server_negotiation.cc:207] Beginning negotiation
0809 07:48:35.837520 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:48:35.837539 (+ 19us) server_negotiation.cc:408] Connection header received
0809 07:48:35.837608 (+ 69us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:48:35.837613 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:48:35.837699 (+ 86us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:48:35.837823 (+ 124us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:48:35.923209 (+ 85386us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.926341 (+ 3132us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.944037 (+ 17696us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:50557: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":26583,"thread_start_us":27171,"threads_started":1}
W20260809 07:48:35.946810 28624 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:48:35.810304 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:32839 (local address 127.26.37.60:40351)
0809 07:48:35.838441 (+ 28137us) server_negotiation.cc:207] Beginning negotiation
0809 07:48:35.838449 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:48:35.838471 (+ 22us) server_negotiation.cc:408] Connection header received
0809 07:48:35.838537 (+ 66us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:48:35.838542 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:48:35.838600 (+ 58us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:48:35.838737 (+ 137us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:48:35.926531 (+ 87794us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.927860 (+ 1329us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.943765 (+ 15905us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:32839: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":28010,"thread_start_us":28508,"threads_started":1}
I20260809 07:48:35.953080 27924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:36.040357 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27769
I20260809 07:48:36.067646 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27945
I20260809 07:48:36.076025 28098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:36.161475 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27945
I20260809 07:48:36.184670 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28114
I20260809 07:48:36.191493 28188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:36.294778 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28114
I20260809 07:48:36.324405 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27327
I20260809 07:48:36.325791 27418 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 07:48:36.328536 27456 consensus_peers.cc:597] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe -> Peer 58dfa892910444ac91b8096dccc8b227 (127.26.37.62:36365): Couldn't send request to peer 58dfa892910444ac91b8096dccc8b227. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:36365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:48:36.430106 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27327
I20260809 07:48:36.458611 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27428
I20260809 07:48:36.459786 27515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:36.567195 27428 master_runner.cc:187] Error getting master registration for 127.26.37.62:36365: Network error: Client connection negotiation failed: client connection to 127.26.37.62:36365: connect: Connection refused (error 111),
I20260809 07:48:36.567283 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27428
I20260809 07:48:36.600214 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36365
--webserver_interface=127.26.37.62
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351 with env {}
W20260809 07:48:36.972733 28649 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:36.973065 28649 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:36.973219 28649 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:36.981277 28649 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:36.981434 28649 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:36.981536 28649 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:36.981611 28649 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:36.991562 28649 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:36.993309 28649 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:36.995270 28649 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:37.003696 28657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:37.003739 28656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:37.004931 28659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:37.005513 28649 server_base.cc:1061] running on GCE node
I20260809 07:48:37.006307 28649 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:37.006516 28649 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:37.006623 28649 hybrid_clock.cc:648] HybridClock initialized: now 1786261717006596 us; error 0 us; skew 500 ppm
I20260809 07:48:37.006971 28649 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:37.009696 28649 webserver.cc:533] Webserver started at http://127.26.37.62:38107/ using document root <none> and password file <none>
I20260809 07:48:37.010401 28649 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:37.010550 28649 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:37.015856 28649 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:48:37.018638 28665 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:37.019989 28649 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:37.020169 28649 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "58dfa892910444ac91b8096dccc8b227"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:37.020875 28649 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:37.048162 28649 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:37.048995 28649 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:37.049285 28649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:37.070048 28649 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36365
I20260809 07:48:37.070092 28735 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36365 every 8 connection(s)
I20260809 07:48:37.071656 28649 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:37.078697 28736 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:37.081029 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28649
I20260809 07:48:37.081248 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:48:37.082140 28736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap starting.
I20260809 07:48:37.082465 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38631
--webserver_interface=127.26.37.61
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351 with env {}
I20260809 07:48:37.102229 28736 log.cc:826] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:37.111125 28736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:37.111673 28736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap complete.
I20260809 07:48:37.118625 28736 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:37.119340 28736 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58dfa892910444ac91b8096dccc8b227, State: Initialized, Role: FOLLOWER
I20260809 07:48:37.120199 28736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:37.121170 28742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:37.121551 28742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:37.122191 28736 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:37.136650 28756 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:37.136926 28756 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: loading cluster ID for follower catalog manager: success
I20260809 07:48:37.142557 28756 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: acquiring CA information for follower catalog manager: success
I20260809 07:48:37.146103 28756 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:37.159649 28649 master_runner.cc:187] Error getting master registration for 127.26.37.61:38631: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38631: connect: Connection refused (error 111),
I20260809 07:48:37.163487 28649 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:48:37.470350 28739 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:37.470760 28739 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:37.470894 28739 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:37.479442 28739 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:37.479569 28739 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:37.479687 28739 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:37.479765 28739 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:37.489959 28739 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:37.491743 28739 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:37.493703 28739 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:37.502364 28772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:37.502892 28771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:37.503638 28774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:37.504962 28739 server_base.cc:1061] running on GCE node
I20260809 07:48:37.505777 28739 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:37.505956 28739 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:37.506049 28739 hybrid_clock.cc:648] HybridClock initialized: now 1786261717506031 us; error 0 us; skew 500 ppm
I20260809 07:48:37.506431 28739 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:37.509284 28739 webserver.cc:533] Webserver started at http://127.26.37.61:41597/ using document root <none> and password file <none>
I20260809 07:48:37.510104 28739 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:37.510212 28739 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:37.515864 28739 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 07:48:37.519104 28780 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:37.520927 28739 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260809 07:48:37.521189 28739 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "5832ac12571c44e5ae371595359e27d9"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:37.522331 28739 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:37.551299 28739 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:37.552625 28739 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:37.553004 28739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:37.578070 28739 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:38631
I20260809 07:48:37.578066 28844 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:38631 every 8 connection(s)
I20260809 07:48:37.580404 28739 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:48:37.587957 28845 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:37.588878 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28739
I20260809 07:48:37.589085 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:48:37.590292 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:40351
--webserver_interface=127.26.37.60
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351 with env {}
I20260809 07:48:37.592978 28845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: Bootstrap starting.
I20260809 07:48:37.611898 28845 log.cc:826] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:37.620895 28845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:37.621560 28845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: Bootstrap complete.
I20260809 07:48:37.628929 28845 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:37.629724 28845 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5832ac12571c44e5ae371595359e27d9, State: Initialized, Role: FOLLOWER
I20260809 07:48:37.630615 28845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:37.631722 28851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:37.632112 28851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:37.632678 28845 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:37.643674 28866 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:37.643874 28866 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: loading cluster ID for follower catalog manager: success
I20260809 07:48:37.648066 28866 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: acquiring CA information for follower catalog manager: success
I20260809 07:48:37.651206 28866 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:37.680796 28739 master_runner.cc:187] Error getting master registration for 127.26.37.60:40351: Network error: Client connection negotiation failed: client connection to 127.26.37.60:40351: connect: Connection refused (error 111),
I20260809 07:48:37.687130 28739 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:48:37.962237 28849 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:37.962560 28849 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:37.962731 28849 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:37.971122 28849 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:37.971216 28849 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:37.971302 28849 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:37.971374 28849 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:37.981508 28849 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:40351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:37.983186 28849 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:37.985093 28849 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:37.993563 28877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:37.993665 28876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:37.995085 28881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:37.995527 28849 server_base.cc:1061] running on GCE node
I20260809 07:48:37.996330 28849 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:37.996532 28849 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:37.996634 28849 hybrid_clock.cc:648] HybridClock initialized: now 1786261717996593 us; error 0 us; skew 500 ppm
I20260809 07:48:37.997041 28849 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:38.000195 28849 webserver.cc:533] Webserver started at http://127.26.37.60:45391/ using document root <none> and password file <none>
I20260809 07:48:38.000989 28849 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:38.001137 28849 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:38.007726 28849 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:48:38.010887 28890 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.012457 28849 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:38.012632 28849 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "3752a28e549149d5bc655d0a712edcbe"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-bw88"
I20260809 07:48:38.013427 28849 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:38.033941 28849 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:38.034988 28849 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:38.035252 28849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:38.057108 28849 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:40351
I20260809 07:48:38.057122 28958 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:40351 every 8 connection(s)
I20260809 07:48:38.058885 28849 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:48:38.066078 28959 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:38.066960 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28849
I20260809 07:48:38.067144 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:48:38.070784 28959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Bootstrap starting.
I20260809 07:48:38.093175 28959 log.cc:826] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:38.102560 28959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:38.103292 28959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: Bootstrap complete.
I20260809 07:48:38.110778 28959 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:38.111516 28959 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3752a28e549149d5bc655d0a712edcbe, State: Initialized, Role: FOLLOWER
I20260809 07:48:38.112320 28959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:38.113754 28965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:38.114104 28965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:38.114957 28959 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:38.127324 28979 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:38.127440 28979 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: loading cluster ID for follower catalog manager: success
I20260809 07:48:38.131733 28979 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: acquiring CA information for follower catalog manager: success
I20260809 07:48:38.135026 28979 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:38.155932 28849 master_runner.cc:264] No leader master found from master 3752a28e549149d5bc655d0a712edcbe
I20260809 07:48:38.156126 28849 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:38.432137 28984 raft_consensus.cc:493] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:38.432447 28984 raft_consensus.cc:515] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:38.434628 28984 leader_election.cc:290] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5832ac12571c44e5ae371595359e27d9 (127.26.37.61:38631), 3752a28e549149d5bc655d0a712edcbe (127.26.37.60:40351)
I20260809 07:48:38.443445 28810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58dfa892910444ac91b8096dccc8b227" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5832ac12571c44e5ae371595359e27d9" is_pre_election: true
I20260809 07:48:38.443841 28926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58dfa892910444ac91b8096dccc8b227" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3752a28e549149d5bc655d0a712edcbe" is_pre_election: true
I20260809 07:48:38.444064 28810 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58dfa892910444ac91b8096dccc8b227 in term 1.
I20260809 07:48:38.444483 28926 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58dfa892910444ac91b8096dccc8b227 in term 1.
I20260809 07:48:38.444736 28672 leader_election.cc:304] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5832ac12571c44e5ae371595359e27d9, 58dfa892910444ac91b8096dccc8b227; no voters:
I20260809 07:48:38.445204 28984 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:48:38.445447 28984 raft_consensus.cc:493] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:38.445576 28984 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:38.451721 28984 raft_consensus.cc:515] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:38.452665 28984 leader_election.cc:290] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [CANDIDATE]: Term 2 election: Requested vote from peers 5832ac12571c44e5ae371595359e27d9 (127.26.37.61:38631), 3752a28e549149d5bc655d0a712edcbe (127.26.37.60:40351)
I20260809 07:48:38.453088 28810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58dfa892910444ac91b8096dccc8b227" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5832ac12571c44e5ae371595359e27d9"
I20260809 07:48:38.453328 28810 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:38.453944 28926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58dfa892910444ac91b8096dccc8b227" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3752a28e549149d5bc655d0a712edcbe"
I20260809 07:48:38.454142 28926 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:38.458777 28810 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58dfa892910444ac91b8096dccc8b227 in term 2.
I20260809 07:48:38.459227 28672 leader_election.cc:304] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5832ac12571c44e5ae371595359e27d9, 58dfa892910444ac91b8096dccc8b227; no voters:
I20260809 07:48:38.459632 28984 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:48:38.460047 28926 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58dfa892910444ac91b8096dccc8b227 in term 2.
I20260809 07:48:38.460148 28984 raft_consensus.cc:697] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 2 LEADER]: Becoming Leader. State: Replica: 58dfa892910444ac91b8096dccc8b227, State: Running, Role: LEADER
I20260809 07:48:38.460705 28984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:38.463228 28989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58dfa892910444ac91b8096dccc8b227. Latest consensus state: current_term: 2 leader_uuid: "58dfa892910444ac91b8096dccc8b227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:38.463604 28989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:38.464350 28991 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:38.466410 28991 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:38.467216 28991 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:38.467325 28991 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:38.468514 28991 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:38.469310 28991 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Loaded TSK: 0
I20260809 07:48:38.470206 28991 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:38.485006 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--time_source=system_unsync with env {}
I20260809 07:48:38.859859 28810 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 2 FOLLOWER]: Refusing update from remote peer 58dfa892910444ac91b8096dccc8b227: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:48:38.861140 28989 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 07:48:38.862202 28993 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:38.862576 28993 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:38.862715 28993 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:48:38.866006 29001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58dfa892910444ac91b8096dccc8b227. Latest consensus state: current_term: 2 leader_uuid: "58dfa892910444ac91b8096dccc8b227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:38.866360 29001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:38.867761 28989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "58dfa892910444ac91b8096dccc8b227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:38.869210 28926 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 2 FOLLOWER]: Refusing update from remote peer 58dfa892910444ac91b8096dccc8b227: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:48:38.870486 28989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:38.870925 28989 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 07:48:38.880090 28993 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:38.880899 28993 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:38.881677 29001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "58dfa892910444ac91b8096dccc8b227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:38.881999 29001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:38.888393 29010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "58dfa892910444ac91b8096dccc8b227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:38.888732 29010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:38.889635 29006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58dfa892910444ac91b8096dccc8b227. Latest consensus state: current_term: 2 leader_uuid: "58dfa892910444ac91b8096dccc8b227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:38.889935 29006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:38.895632 28993 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:38.897488 28993 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:38.899631 28993 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:38.910123 29013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:38.910081 29012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:38.911012 29015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:38.911662 28993 server_base.cc:1061] running on GCE node
I20260809 07:48:38.912253 28993 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:38.912442 28993 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:38.912542 28993 hybrid_clock.cc:648] HybridClock initialized: now 1786261718912501 us; error 0 us; skew 500 ppm
I20260809 07:48:38.912875 28993 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:38.915261 28993 webserver.cc:533] Webserver started at http://127.26.37.1:45659/ using document root <none> and password file <none>
I20260809 07:48:38.916437 28993 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:38.916620 28993 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:38.922730 28993 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 07:48:38.925835 29021 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.927271 28993 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:48:38.927477 28993 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:38.928242 28993 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:38.955521 28993 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:38.956501 28993 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:38.956815 28993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:38.957886 28993 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:38.959739 28993 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:38.959863 28993 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.960012 28993 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:38.960085 28993 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.995329 28993 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39379
I20260809 07:48:38.995393 29164 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39379 every 8 connection(s)
I20260809 07:48:38.997155 28993 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:39.000397 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28993
I20260809 07:48:39.000610 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:39.006744 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--time_source=system_unsync with env {}
I20260809 07:48:39.033208 29165 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:39.034106 29165 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:39.035086 29165 heartbeater.cc:507] Master 127.26.37.60:40351 requested a full tablet report, sending...
I20260809 07:48:39.036360 29166 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:39.036568 29166 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:39.037029 29166 heartbeater.cc:507] Master 127.26.37.62:36365 requested a full tablet report, sending...
I20260809 07:48:39.038022 28909 ts_manager.cc:194] Registered new tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:39379)
I20260809 07:48:39.038872 28690 ts_manager.cc:194] Registered new tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:39379)
I20260809 07:48:39.040815 28690 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:52033
I20260809 07:48:39.046139 29167 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38631
I20260809 07:48:39.046331 29167 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:39.046684 29167 heartbeater.cc:507] Master 127.26.37.61:38631 requested a full tablet report, sending...
I20260809 07:48:39.048617 28800 ts_manager.cc:194] Registered new tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:39379)
I20260809 07:48:39.256218 29004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "58dfa892910444ac91b8096dccc8b227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:39.256565 29004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: LEADER
W20260809 07:48:39.379868 29177 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:39.380228 29177 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:39.380378 29177 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:39.388674 29177 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:39.388882 29177 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:48:39.398504 29177 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:39.400228 29177 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:39.402271 29177 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:39.411890 29184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:39.411862 29185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:39.412302 29187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:39.413030 29177 server_base.cc:1061] running on GCE node
I20260809 07:48:39.413636 29177 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:39.413786 29177 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:39.413851 29177 hybrid_clock.cc:648] HybridClock initialized: now 1786261719413832 us; error 0 us; skew 500 ppm
I20260809 07:48:39.414096 29177 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:39.416188 29177 webserver.cc:533] Webserver started at http://127.26.37.2:38753/ using document root <none> and password file <none>
I20260809 07:48:39.417016 29177 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:39.417135 29177 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:39.422768 29177 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 07:48:39.425679 29194 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:39.427171 29177 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:48:39.427338 29177 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "9aff321a559b4c6ba39d521c2a3eef98"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:39.428112 29177 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:39.447536 29177 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:39.448453 29177 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:39.448760 29177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:39.449867 29177 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:39.451637 29177 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:39.451720 29177 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:39.451866 29177 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:39.451929 29177 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:39.487141 29177 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:37503
I20260809 07:48:39.487185 29345 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:37503 every 8 connection(s)
I20260809 07:48:39.488723 29177 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:39.498688 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29177
I20260809 07:48:39.498880 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:39.503501 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--time_source=system_unsync with env {}
I20260809 07:48:39.506260 29346 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:39.506284 29349 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38631
I20260809 07:48:39.506669 29346 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:39.506675 29349 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:39.507676 29346 heartbeater.cc:507] Master 127.26.37.60:40351 requested a full tablet report, sending...
I20260809 07:48:39.507720 29349 heartbeater.cc:507] Master 127.26.37.61:38631 requested a full tablet report, sending...
I20260809 07:48:39.509506 28909 ts_manager.cc:194] Registered new tserver with Master: 9aff321a559b4c6ba39d521c2a3eef98 (127.26.37.2:37503)
I20260809 07:48:39.509505 28800 ts_manager.cc:194] Registered new tserver with Master: 9aff321a559b4c6ba39d521c2a3eef98 (127.26.37.2:37503)
I20260809 07:48:39.514189 29347 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:39.514340 29347 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:39.514828 29347 heartbeater.cc:507] Master 127.26.37.62:36365 requested a full tablet report, sending...
I20260809 07:48:39.516139 28690 ts_manager.cc:194] Registered new tserver with Master: 9aff321a559b4c6ba39d521c2a3eef98 (127.26.37.2:37503)
I20260809 07:48:39.517272 28690 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:34441
W20260809 07:48:39.884822 29356 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:39.885138 29356 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:39.885210 29356 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:39.893859 29356 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:39.894012 29356 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:48:39.903568 29356 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:39.905102 29356 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:39.906956 29356 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:39.916332 29362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:39.916337 29363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:39.917201 29365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:39.917802 29356 server_base.cc:1061] running on GCE node
I20260809 07:48:39.918522 29356 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:39.918735 29356 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:39.918826 29356 hybrid_clock.cc:648] HybridClock initialized: now 1786261719918805 us; error 0 us; skew 500 ppm
I20260809 07:48:39.919199 29356 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:39.921391 29356 webserver.cc:533] Webserver started at http://127.26.37.3:34463/ using document root <none> and password file <none>
I20260809 07:48:39.922165 29356 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:39.922309 29356 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:39.927925 29356 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 07:48:39.930682 29370 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:39.932041 29356 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:39.932222 29356 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "ee2286de73d64942935bc447e02088d3"
format_stamp: "Formatted at 2026-08-09 07:48:33 on dist-test-slave-bw88"
I20260809 07:48:39.932940 29356 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:39.952293 29356 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:39.953177 29356 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:39.953572 29356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:39.954535 29356 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:39.956194 29356 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:39.956313 29356 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:39.956457 29356 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:39.956533 29356 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:39.993014 29356 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:34063
I20260809 07:48:39.993052 29517 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:34063 every 8 connection(s)
I20260809 07:48:39.994706 29356 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:40.000018 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29356
I20260809 07:48:40.000284 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:40.014519 29518 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:40.014599 29519 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:40.014633 29520 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38631
I20260809 07:48:40.015038 29520 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:40.015035 29518 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:40.015035 29519 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:40.016194 29518 heartbeater.cc:507] Master 127.26.37.60:40351 requested a full tablet report, sending...
I20260809 07:48:40.016215 29519 heartbeater.cc:507] Master 127.26.37.62:36365 requested a full tablet report, sending...
I20260809 07:48:40.016930 29520 heartbeater.cc:507] Master 127.26.37.61:38631 requested a full tablet report, sending...
I20260809 07:48:40.018224 28690 ts_manager.cc:194] Registered new tserver with Master: ee2286de73d64942935bc447e02088d3 (127.26.37.3:34063)
I20260809 07:48:40.018338 28800 ts_manager.cc:194] Registered new tserver with Master: ee2286de73d64942935bc447e02088d3 (127.26.37.3:34063)
I20260809 07:48:40.018630 28909 ts_manager.cc:194] Registered new tserver with Master: ee2286de73d64942935bc447e02088d3 (127.26.37.3:34063)
I20260809 07:48:40.019311 28690 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:57379
I20260809 07:48:40.027768 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:48:40.035655 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:48:40.035893 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28649
I20260809 07:48:40.036958 28731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:40.043892 29166 heartbeater.cc:499] Master 127.26.37.62:36365 was elected leader, sending a full tablet report...
I20260809 07:48:40.146596 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28649
I20260809 07:48:40.176259 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36365
--webserver_interface=127.26.37.62
--webserver_port=38107
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351 with env {}
I20260809 07:48:40.520289 29347 heartbeater.cc:499] Master 127.26.37.62:36365 was elected leader, sending a full tablet report...
W20260809 07:48:40.521799 29527 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:40.522193 29527 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:40.522154 29347 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:36365 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:36365: connect: Connection refused (error 111)
W20260809 07:48:40.522338 29527 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:40.531137 29527 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:40.531289 29527 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:40.531358 29527 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:40.531531 29527 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:40.543352 29527 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=38107
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:40.545148 29527 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:40.547152 29527 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:40.555995 29535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:40.556715 29539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:40.556910 29536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:40.557579 29527 server_base.cc:1061] running on GCE node
I20260809 07:48:40.558447 29527 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:40.558655 29527 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:40.558746 29527 hybrid_clock.cc:648] HybridClock initialized: now 1786261720558727 us; error 0 us; skew 500 ppm
I20260809 07:48:40.559139 29527 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:40.562223 29527 webserver.cc:533] Webserver started at http://127.26.37.62:38107/ using document root <none> and password file <none>
I20260809 07:48:40.563066 29527 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:40.563210 29527 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:40.569042 29527 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:48:40.572223 29545 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.573771 29527 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:40.573980 29527 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "58dfa892910444ac91b8096dccc8b227"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-bw88"
I20260809 07:48:40.574779 29527 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:40.595168 29527 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:40.596311 29527 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:40.596663 29527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:40.618335 29527 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36365
I20260809 07:48:40.618330 29616 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36365 every 8 connection(s)
I20260809 07:48:40.620311 29527 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:40.626726 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29527
I20260809 07:48:40.627092 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28993
I20260809 07:48:40.627835 29617 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:40.632733 29617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap starting.
I20260809 07:48:40.637790 29159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:40.652529 29617 log.cc:826] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:40.662236 29617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:40.662809 29617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: Bootstrap complete.
I20260809 07:48:40.669790 29617 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:40.670560 29617 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58dfa892910444ac91b8096dccc8b227, State: Initialized, Role: FOLLOWER
I20260809 07:48:40.671278 29617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:40.672187 29625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:40.672528 29625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:40.673192 29617 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:40.684631 29639 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:40.684873 29639 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: loading cluster ID for follower catalog manager: success
I20260809 07:48:40.690393 29639 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: acquiring CA information for follower catalog manager: success
I20260809 07:48:40.693800 29639 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:40.725069 29527 master_runner.cc:264] No leader master found from master 58dfa892910444ac91b8096dccc8b227
I20260809 07:48:40.725244 29527 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:40.753036 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28993
I20260809 07:48:40.780391 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39379
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--webserver_port=45659
--webserver_interface=127.26.37.1
--time_source=system_unsync with env {}
I20260809 07:48:41.021467 29519 heartbeater.cc:499] Master 127.26.37.62:36365 was elected leader, sending a full tablet report...
I20260809 07:48:41.032681 29570 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ee2286de73d64942935bc447e02088d3" instance_seqno: 1786261719984547) as {username='slave'} at 127.26.37.3:41313; Asking this server to re-register.
I20260809 07:48:41.033741 29519 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:41.035720 29570 ts_manager.cc:194] Registered new tserver with Master: ee2286de73d64942935bc447e02088d3 (127.26.37.3:34063)
W20260809 07:48:41.170491 29647 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:41.170858 29647 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:41.171013 29647 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:41.179921 29647 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:41.180190 29647 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:41.190464 29647 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39379
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45659
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:41.192200 29647 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:41.194245 29647 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:41.204736 29655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.204704 29656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.205595 29659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:41.205876 29647 server_base.cc:1061] running on GCE node
I20260809 07:48:41.206562 29647 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:41.206768 29647 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:41.206887 29647 hybrid_clock.cc:648] HybridClock initialized: now 1786261721206842 us; error 0 us; skew 500 ppm
I20260809 07:48:41.207297 29647 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:41.209831 29647 webserver.cc:533] Webserver started at http://127.26.37.1:45659/ using document root <none> and password file <none>
I20260809 07:48:41.210683 29647 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:41.210831 29647 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:41.217079 29647 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:48:41.220266 29665 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.221933 29647 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:41.222122 29647 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:41.222991 29647 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:41.236840 29647 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:41.237804 29647 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:41.238164 29647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:41.239208 29647 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:41.241282 29647 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:41.241454 29647 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 07:48:41.241564 29647 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:41.241611 29647 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.258944 29735 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58dfa892910444ac91b8096dccc8b227)
I20260809 07:48:41.259243 29735 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:41.262269 29735 leader_election.cc:290] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58dfa892910444ac91b8096dccc8b227 (127.26.37.62:36365), 3752a28e549149d5bc655d0a712edcbe (127.26.37.60:40351)
I20260809 07:48:41.274291 28926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5832ac12571c44e5ae371595359e27d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3752a28e549149d5bc655d0a712edcbe" is_pre_election: true
I20260809 07:48:41.275599 29582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5832ac12571c44e5ae371595359e27d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58dfa892910444ac91b8096dccc8b227" is_pre_election: true
I20260809 07:48:41.276274 29582 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5832ac12571c44e5ae371595359e27d9 in term 2.
I20260809 07:48:41.276998 28783 leader_election.cc:304] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5832ac12571c44e5ae371595359e27d9, 58dfa892910444ac91b8096dccc8b227; no voters: 3752a28e549149d5bc655d0a712edcbe
I20260809 07:48:41.277763 29735 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:48:41.277902 29735 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58dfa892910444ac91b8096dccc8b227)
I20260809 07:48:41.278023 29735 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:48:41.280674 29735 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:41.281729 29735 leader_election.cc:290] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [CANDIDATE]: Term 3 election: Requested vote from peers 58dfa892910444ac91b8096dccc8b227 (127.26.37.62:36365), 3752a28e549149d5bc655d0a712edcbe (127.26.37.60:40351)
I20260809 07:48:41.282251 29582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5832ac12571c44e5ae371595359e27d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58dfa892910444ac91b8096dccc8b227"
I20260809 07:48:41.282366 28926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5832ac12571c44e5ae371595359e27d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3752a28e549149d5bc655d0a712edcbe"
I20260809 07:48:41.282500 29582 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:48:41.288353 29582 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5832ac12571c44e5ae371595359e27d9 in term 3.
I20260809 07:48:41.288874 28783 leader_election.cc:304] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5832ac12571c44e5ae371595359e27d9, 58dfa892910444ac91b8096dccc8b227; no voters: 3752a28e549149d5bc655d0a712edcbe
I20260809 07:48:41.288937 29647 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39379
I20260809 07:48:41.289028 29815 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39379 every 8 connection(s)
I20260809 07:48:41.289309 29735 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:48:41.289635 29816 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58dfa892910444ac91b8096dccc8b227)
I20260809 07:48:41.289904 29816 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:41.290107 29735 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [term 3 LEADER]: Becoming Leader. State: Replica: 5832ac12571c44e5ae371595359e27d9, State: Running, Role: LEADER
I20260809 07:48:41.290688 29647 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:41.290607 29735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } }
I20260809 07:48:41.294086 29816 leader_election.cc:290] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58dfa892910444ac91b8096dccc8b227 (127.26.37.62:36365), 5832ac12571c44e5ae371595359e27d9 (127.26.37.61:38631)
I20260809 07:48:41.297540 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29647
I20260809 07:48:41.297976 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29647
I20260809 07:48:41.298411 28810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3752a28e549149d5bc655d0a712edcbe" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5832ac12571c44e5ae371595359e27d9" is_pre_election: true
I20260809 07:48:41.304780 29817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5832ac12571c44e5ae371595359e27d9. Latest consensus state: current_term: 3 leader_uuid: "5832ac12571c44e5ae371595359e27d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:41.305147 29817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:41.341130 29582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3752a28e549149d5bc655d0a712edcbe" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58dfa892910444ac91b8096dccc8b227" is_pre_election: true
I20260809 07:48:41.341956 29582 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3752a28e549149d5bc655d0a712edcbe in current term 3: Already voted for candidate 5832ac12571c44e5ae371595359e27d9 in this term.
I20260809 07:48:41.343223 28894 leader_election.cc:304] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3752a28e549149d5bc655d0a712edcbe; no voters: 5832ac12571c44e5ae371595359e27d9, 58dfa892910444ac91b8096dccc8b227
I20260809 07:48:41.344241 29816 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:48:41.348624 29816 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:48:41.369688 29830 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:41.373118 29830 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:41.374490 29830 catalog_manager.cc:1295] Loaded cluster ID: 78326fb3484f41ee80e7379701a3dadc
I20260809 07:48:41.374697 29830 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:41.376453 29830 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:41.377728 29830 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9: Loaded TSK: 0
I20260809 07:48:41.378847 29830 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:41.385293 29810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:41.468986 29818 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:41.468986 29822 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38631
I20260809 07:48:41.469743 29818 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:41.469283 29819 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:41.469974 29819 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:41.469754 29822 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:41.470572 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29647
I20260809 07:48:41.497970 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39379
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--webserver_port=45659
--webserver_interface=127.26.37.1
--time_source=system_unsync
--fault_before_start=1.0 with env {}
I20260809 07:48:41.530187 29347 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:41.530428 29347 heartbeater.cc:499] Master 127.26.37.62:36365 was elected leader, sending a full tablet report...
I20260809 07:48:41.531461 29570 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "9aff321a559b4c6ba39d521c2a3eef98" instance_seqno: 1786261719478849) as {username='slave'} at 127.26.37.2:55297; Asking this server to re-register.
I20260809 07:48:41.532100 29347 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:41.533411 29570 ts_manager.cc:194] Registered new tserver with Master: 9aff321a559b4c6ba39d521c2a3eef98 (127.26.37.2:37503)
I20260809 07:48:41.723121 29582 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [term 3 FOLLOWER]: Refusing update from remote peer 5832ac12571c44e5ae371595359e27d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:48:41.724195 29817 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:48:41.728688 29839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5832ac12571c44e5ae371595359e27d9. Latest consensus state: current_term: 3 leader_uuid: "5832ac12571c44e5ae371595359e27d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:41.729067 29839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:41.730275 29735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "5832ac12571c44e5ae371595359e27d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:41.730626 29735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:41.731606 28926 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [term 3 FOLLOWER]: Refusing update from remote peer 5832ac12571c44e5ae371595359e27d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:48:41.732630 29735 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:48:41.744370 29839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "5832ac12571c44e5ae371595359e27d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:41.744661 29839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58dfa892910444ac91b8096dccc8b227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:41.744784 29816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5832ac12571c44e5ae371595359e27d9. Latest consensus state: current_term: 3 leader_uuid: "5832ac12571c44e5ae371595359e27d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:41.744745 29848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "5832ac12571c44e5ae371595359e27d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:41.745060 29816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:41.745133 29848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3752a28e549149d5bc655d0a712edcbe [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:48:41.867660 29837 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:41.868068 29837 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:41.868249 29837 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:48:41.868357 29837 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:41.877070 29837 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:41.877314 29837 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:41.887575 29837 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39379
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45659
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:41.889408 29837 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:41.891196 29837 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:41.901064 29857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.901291 29855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.902124 29861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:41.902462 29837 server_base.cc:1061] running on GCE node
I20260809 07:48:41.903028 29837 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:41.903228 29837 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:41.903347 29837 hybrid_clock.cc:648] HybridClock initialized: now 1786261721903290 us; error 0 us; skew 500 ppm
I20260809 07:48:41.903656 29837 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:41.906031 29837 webserver.cc:533] Webserver started at http://127.26.37.1:45659/ using document root <none> and password file <none>
I20260809 07:48:41.906801 29837 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:41.906958 29837 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:41.912652 29837 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:48:41.915462 29867 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.916935 29837 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:41.917116 29837 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:41.917985 29837 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:41.937722 29837 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:41.938778 29837 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:41.939065 29837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:41.940070 29837 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:41.941812 29837 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:41.941933 29837 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.942070 29837 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:41.942167 29837 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:48:41.942531 29837 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:48:41.942662 29837 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:48:41.942734 29837 fault_injection.cc:59] injecting fault for kudu (pid 29837): FLAGS_fault_before_start (process will exit)
I20260809 07:48:41.964041 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39379
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--webserver_port=45659
--webserver_interface=127.26.37.1
--time_source=system_unsync with env {}
I20260809 07:48:42.270629 29817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "5832ac12571c44e5ae371595359e27d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58dfa892910444ac91b8096dccc8b227" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36365 } } peers { permanent_uuid: "5832ac12571c44e5ae371595359e27d9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38631 } } peers { permanent_uuid: "3752a28e549149d5bc655d0a712edcbe" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 40351 } } }
I20260809 07:48:42.271047 29817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5832ac12571c44e5ae371595359e27d9 [sys.catalog]: This master's current role is: LEADER
W20260809 07:48:42.318326 29876 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:42.318707 29876 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:42.318837 29876 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:42.327410 29876 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:42.327574 29876 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:42.337657 29876 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39379
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45659
--tserver_master_addrs=127.26.37.62:36365,127.26.37.61:38631,127.26.37.60:40351
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:42.339401 29876 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:42.341144 29876 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:42.350980 29886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:42.350941 29887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:42.351897 29889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:42.352308 29876 server_base.cc:1061] running on GCE node
I20260809 07:48:42.352888 29876 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:48:42.353039 29876 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:48:42.353106 29876 hybrid_clock.cc:648] HybridClock initialized: now 1786261722353087 us; error 0 us; skew 500 ppm
I20260809 07:48:42.353407 29876 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:42.355629 29876 webserver.cc:533] Webserver started at http://127.26.37.1:45659/ using document root <none> and password file <none>
I20260809 07:48:42.356346 29876 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:42.356446 29876 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:42.362156 29876 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 07:48:42.365016 29895 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:42.366489 29876 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:48:42.366659 29876 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "73ac8fc9bca740649ee5a358103bc122"
format_stamp: "Formatted at 2026-08-09 07:48:32 on dist-test-slave-bw88"
I20260809 07:48:42.367419 29876 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:42.400372 29876 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:42.401400 29876 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:42.401726 29876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:42.402799 29876 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:42.404481 29876 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:42.404556 29876 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:42.404723 29876 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:42.404775 29876 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:42.440187 29876 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39379
I20260809 07:48:42.440234 30032 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39379 every 8 connection(s)
I20260809 07:48:42.441793 29876 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:42.446514 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29876
I20260809 07:48:42.446946 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29876
I20260809 07:48:42.464870 30033 heartbeater.cc:344] Connected to a master server at 127.26.37.60:40351
I20260809 07:48:42.465243 30033 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:42.466573 30033 heartbeater.cc:507] Master 127.26.37.60:40351 requested a full tablet report, sending...
I20260809 07:48:42.469060 28909 ts_manager.cc:194] Re-registered known tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:39379)
I20260809 07:48:42.471653 30027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:42.515980 29349 heartbeater.cc:499] Master 127.26.37.61:38631 was elected leader, sending a full tablet report...
I20260809 07:48:42.557622 30034 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36365
I20260809 07:48:42.557891 30034 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:42.558722 30034 heartbeater.cc:507] Master 127.26.37.62:36365 requested a full tablet report, sending...
I20260809 07:48:42.559326 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29876
I20260809 07:48:42.564638 29570 ts_manager.cc:194] Registered new tserver with Master: 73ac8fc9bca740649ee5a358103bc122 (127.26.37.1:39379)
W20260809 07:48:42.584760 29551 connection.cc:573] server connection from 127.26.37.1:48435 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:48:42.585479 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29177
W20260809 07:48:42.586223 30037 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:48:42.445144 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:48119 (local address 127.26.37.61:38631)
0809 07:48:42.445678 (+ 534us) server_negotiation.cc:207] Beginning negotiation
0809 07:48:42.445685 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:48:42.459895 (+ 14210us) server_negotiation.cc:408] Connection header received
0809 07:48:42.460018 (+ 123us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:48:42.460022 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:48:42.460122 (+ 100us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:48:42.460250 (+ 128us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:48:42.461577 (+ 1327us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.462688 (+ 1111us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.558509 (+ 95821us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.559072 (+ 563us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:48:42.585014 (+ 25942us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:48119: BlockingRecv error: recv got EOF from 127.26.37.1:48119 (error 108)
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":477,"threads_started":1}
I20260809 07:48:42.594228 29340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:42.688000 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29177
I20260809 07:48:42.711570 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29356
I20260809 07:48:42.720196 29513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:42.808389 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29356
I20260809 07:48:42.832355 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29527
I20260809 07:48:42.839023 29609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:42.944501 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29527
I20260809 07:48:42.973692 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28739
I20260809 07:48:42.974947 28839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:43.086946 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28739
I20260809 07:48:43.117527 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28849
I20260809 07:48:43.118885 28953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:43.224485 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28849
[ OK ] ExternalMiniClusterTest.TestBasicOperation/0 (13253 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/1
2026-08-09T07:48:43Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:43Z Disabled control of system clock
I20260809 07:48:43.293962 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36285
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025 with env {}
W20260809 07:48:43.629194 30063 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:43.629710 30063 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:43.629868 30063 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:43.638610 30063 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:43.638710 30063 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:43.638818 30063 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:43.638916 30063 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:43.649389 30063 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:43.651300 30063 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:43.653205 30063 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:43.661756 30070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:43.662195 30071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:43.662940 30073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:43.663376 30063 server_base.cc:1061] running on GCE node
I20260809 07:48:43.664136 30063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:43.665560 30063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:43.666790 30063 hybrid_clock.cc:648] HybridClock initialized: now 1786261723666742 us; error 61 us; skew 500 ppm
I20260809 07:48:43.667140 30063 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:43.669305 30063 webserver.cc:533] Webserver started at http://127.26.37.62:34945/ using document root <none> and password file <none>
I20260809 07:48:43.670097 30063 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:43.670238 30063 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:43.670639 30063 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:43.673179 30063 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "cbf5887cb00342e19f5e051f71bb04a2"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-bw88"
I20260809 07:48:43.674036 30063 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "cbf5887cb00342e19f5e051f71bb04a2"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-bw88"
I20260809 07:48:43.679584 30063 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:48:43.683054 30082 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:43.684391 30063 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:43.684608 30063 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cbf5887cb00342e19f5e051f71bb04a2"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-bw88"
I20260809 07:48:43.684832 30063 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:43.702061 30063 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:43.702919 30063 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:43.703212 30063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:43.724001 30063 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36285
I20260809 07:48:43.724025 30149 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36285 every 8 connection(s)
I20260809 07:48:43.725675 30063 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:43.730046 30150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:43.732411 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30063
I20260809 07:48:43.732640 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:48:43.733747 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:42757
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025 with env {}
I20260809 07:48:43.736078 30150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:43.753196 30150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:43.754602 30084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42757: connect: Connection refused (error 111)
W20260809 07:48:43.756385 30150 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42757: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42757: connect: Connection refused (error 111)
I20260809 07:48:43.805765 30150 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } attempt: 1
W20260809 07:48:43.808087 30150 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42757: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42757: connect: Connection refused (error 111)
I20260809 07:48:43.876367 30150 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } attempt: 2
W20260809 07:48:43.878584 30150 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42757: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42757: connect: Connection refused (error 111)
I20260809 07:48:43.969872 30150 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } attempt: 3
W20260809 07:48:43.972244 30150 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42757: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42757: connect: Connection refused (error 111)
W20260809 07:48:44.085839 30156 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:44.086199 30156 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:44.086349 30156 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:44.094803 30156 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:44.094889 30156 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:44.094980 30156 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:44.095079 30156 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:44.104766 30156 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:42757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:44.106462 30156 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:44.108364 30156 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:48:44.115545 30150 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } attempt: 4
W20260809 07:48:44.116662 30167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:44.117884 30168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:44.118366 30170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:44.118541 30150 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42757: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42757: connect: Connection refused (error 111)
I20260809 07:48:44.118844 30156 server_base.cc:1061] running on GCE node
I20260809 07:48:44.119714 30156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:44.121230 30156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:44.122489 30156 hybrid_clock.cc:648] HybridClock initialized: now 1786261724122416 us; error 58 us; skew 500 ppm
I20260809 07:48:44.122923 30156 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:44.125380 30156 webserver.cc:533] Webserver started at http://127.26.37.61:33069/ using document root <none> and password file <none>
I20260809 07:48:44.126202 30156 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:44.126350 30156 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:44.126791 30156 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:44.129455 30156 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "950ff7120dbc4fbebd2065da8b3adc57"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-bw88"
I20260809 07:48:44.130633 30156 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "950ff7120dbc4fbebd2065da8b3adc57"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-bw88"
I20260809 07:48:44.136428 30156 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:48:44.140033 30179 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:44.141683 30156 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:44.141880 30156 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "950ff7120dbc4fbebd2065da8b3adc57"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-bw88"
I20260809 07:48:44.142123 30156 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:44.169576 30156 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:44.170562 30156 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:44.170872 30156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:44.192322 30156 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:42757
I20260809 07:48:44.192341 30241 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:42757 every 8 connection(s)
I20260809 07:48:44.194005 30156 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:48:44.196640 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30156
I20260809 07:48:44.196843 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:48:44.198343 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:35025
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025 with env {}
I20260809 07:48:44.199563 30242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:44.207729 30242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:44.221984 30242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:44.231562 30242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:44.232728 30181 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
W20260809 07:48:44.234395 30242 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
I20260809 07:48:44.283761 30242 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } attempt: 1
W20260809 07:48:44.286208 30242 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
I20260809 07:48:44.354460 30242 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } attempt: 2
W20260809 07:48:44.356671 30242 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
I20260809 07:48:44.418025 30150 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } attempt: 5
I20260809 07:48:44.425451 30150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:44.427523 30150 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
I20260809 07:48:44.447907 30242 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } attempt: 3
W20260809 07:48:44.450224 30242 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
I20260809 07:48:44.478820 30150 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } attempt: 1
W20260809 07:48:44.480875 30150 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
I20260809 07:48:44.549141 30150 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } attempt: 2
W20260809 07:48:44.551237 30150 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
W20260809 07:48:44.551000 30248 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:44.551463 30248 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:44.551584 30248 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:44.560017 30248 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:44.560096 30248 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:44.560197 30248 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:44.560276 30248 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:44.570223 30248 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:35025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:44.571928 30248 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:44.573859 30248 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:44.582505 30257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:44.582479 30259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:44.583742 30263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:44.584623 30248 server_base.cc:1061] running on GCE node
I20260809 07:48:44.585438 30248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:44.586923 30248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:44.588140 30248 hybrid_clock.cc:648] HybridClock initialized: now 1786261724588092 us; error 48 us; skew 500 ppm
I20260809 07:48:44.588542 30248 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:44.591078 30248 webserver.cc:533] Webserver started at http://127.26.37.60:35903/ using document root <none> and password file <none>
I20260809 07:48:44.591905 30248 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:44.592061 30248 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:44.592454 30248 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:44.593434 30242 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } attempt: 4
W20260809 07:48:44.595475 30242 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111)
I20260809 07:48:44.595193 30248 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "5a028f1f75b94a56adade49eb2e6d92d"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-bw88"
I20260809 07:48:44.596019 30248 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "5a028f1f75b94a56adade49eb2e6d92d"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-bw88"
I20260809 07:48:44.601228 30248 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:48:44.604557 30269 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:44.605865 30248 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:44.606040 30248 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "5a028f1f75b94a56adade49eb2e6d92d"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-bw88"
I20260809 07:48:44.606236 30248 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:44.627787 30248 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:44.628728 30248 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:44.629063 30248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:44.650151 30248 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:35025
I20260809 07:48:44.650158 30338 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:35025 every 8 connection(s)
I20260809 07:48:44.651731 30248 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:48:44.655936 30339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:44.657452 30150 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } attempt: 3
I20260809 07:48:44.661687 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30248
I20260809 07:48:44.661897 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:48:44.664464 30339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:44.677640 30339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:44.679862 30150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap starting.
I20260809 07:48:44.683985 30150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:44.685042 30339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:44.685504 30150 log.cc:826] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:44.688619 30150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: No bootstrap required, opened a new log
I20260809 07:48:44.695272 30150 raft_consensus.cc:359] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:44.695834 30150 raft_consensus.cc:385] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:44.696033 30150 raft_consensus.cc:740] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf5887cb00342e19f5e051f71bb04a2, State: Initialized, Role: FOLLOWER
I20260809 07:48:44.696630 30150 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:44.696779 30339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Bootstrap starting.
I20260809 07:48:44.697675 30350 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:44.697978 30350 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:44.698835 30150 sys_catalog.cc:565] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:44.700019 30339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:44.701471 30339 log.cc:826] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:44.704270 30339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: No bootstrap required, opened a new log
W20260809 07:48:44.705801 30088 tablet.cc:2406] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:44.711951 30339 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:44.712345 30339 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:44.712520 30339 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a028f1f75b94a56adade49eb2e6d92d, State: Initialized, Role: FOLLOWER
I20260809 07:48:44.713254 30339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
W20260809 07:48:44.713646 30370 catalog_manager.cc:1594] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:44.713884 30370 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:44.714666 30373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:44.715060 30373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:44.715912 30339 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:44.708966 30063 master_runner.cc:187] Error getting master registration for 127.26.37.61:42757: OK, instance_id { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" instance_seqno: 1786261724186546 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:48:44.716805 30063 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:44.728914 30248 master_runner.cc:187] Error getting master registration for 127.26.37.61:42757: OK, instance_id { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" instance_seqno: 1786261724186546 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:48:44.735430 30248 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:48:44.740203 30387 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:44.740342 30387 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:44.867683 30350 raft_consensus.cc:493] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:44.868018 30350 raft_consensus.cc:515] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:44.870105 30350 leader_election.cc:290] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 950ff7120dbc4fbebd2065da8b3adc57 (127.26.37.61:42757), 5a028f1f75b94a56adade49eb2e6d92d (127.26.37.60:35025)
I20260809 07:48:44.871577 30312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbf5887cb00342e19f5e051f71bb04a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a028f1f75b94a56adade49eb2e6d92d" is_pre_election: true
I20260809 07:48:44.872202 30312 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf5887cb00342e19f5e051f71bb04a2 in term 0.
I20260809 07:48:44.873092 30084 leader_election.cc:304] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a028f1f75b94a56adade49eb2e6d92d, cbf5887cb00342e19f5e051f71bb04a2; no voters:
I20260809 07:48:44.874042 30350 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:44.874225 30350 raft_consensus.cc:493] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:44.874339 30350 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:44.877913 30350 raft_consensus.cc:515] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:44.879901 30312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbf5887cb00342e19f5e051f71bb04a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a028f1f75b94a56adade49eb2e6d92d"
I20260809 07:48:44.880131 30312 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:44.883858 30312 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf5887cb00342e19f5e051f71bb04a2 in term 1.
W20260809 07:48:44.884366 30279 tablet.cc:2406] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:44.885474 30084 leader_election.cc:304] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a028f1f75b94a56adade49eb2e6d92d, cbf5887cb00342e19f5e051f71bb04a2; no voters:
I20260809 07:48:44.886231 30350 leader_election.cc:290] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [CANDIDATE]: Term 1 election: Requested vote from peers 950ff7120dbc4fbebd2065da8b3adc57 (127.26.37.61:42757), 5a028f1f75b94a56adade49eb2e6d92d (127.26.37.60:35025)
I20260809 07:48:44.886641 30390 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:44.887059 30390 raft_consensus.cc:697] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 1 LEADER]: Becoming Leader. State: Replica: cbf5887cb00342e19f5e051f71bb04a2, State: Running, Role: LEADER
I20260809 07:48:44.888226 30390 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:44.894271 30350 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbf5887cb00342e19f5e051f71bb04a2. Latest consensus state: current_term: 1 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:44.894541 30350 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:44.894874 30242 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } attempt: 5
I20260809 07:48:44.903227 30392 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:44.905726 30392 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:44.914913 30242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: Bootstrap starting.
I20260809 07:48:44.923439 30312 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 1 FOLLOWER]: Refusing update from remote peer cbf5887cb00342e19f5e051f71bb04a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:44.925812 30242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:44.926901 30390 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:44.928697 30242 log.cc:826] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:44.942970 30373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbf5887cb00342e19f5e051f71bb04a2. Latest consensus state: current_term: 1 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:44.943274 30373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:44.946592 30393 mvcc.cc:204] Tried to move back new op lower bound from 7316528025269297152 to 7316528025152593920. Current Snapshot: MvccSnapshot[applied={T|T < 7316528025269297152}]
I20260809 07:48:44.949653 30395 mvcc.cc:204] Tried to move back new op lower bound from 7316528025269297152 to 7316528025152593920. Current Snapshot: MvccSnapshot[applied={T|T < 7316528025269297152}]
I20260809 07:48:44.951123 30373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:44.951380 30373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:44.951578 30242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: No bootstrap required, opened a new log
I20260809 07:48:44.958012 30390 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:44.958313 30390 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:44.959868 30392 catalog_manager.cc:1383] Generated new cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:44.959643 30242 raft_consensus.cc:359] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:44.959982 30392 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:44.959995 30242 raft_consensus.cc:385] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:44.960137 30242 raft_consensus.cc:740] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950ff7120dbc4fbebd2065da8b3adc57, State: Initialized, Role: FOLLOWER
I20260809 07:48:44.962144 30242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:44.974777 30242 sys_catalog.cc:565] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:44.974521 30210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbf5887cb00342e19f5e051f71bb04a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "950ff7120dbc4fbebd2065da8b3adc57" is_pre_election: true
I20260809 07:48:44.974727 30209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbf5887cb00342e19f5e051f71bb04a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "950ff7120dbc4fbebd2065da8b3adc57"
I20260809 07:48:44.976011 30208 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:44.979730 30208 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 1 FOLLOWER]: Refusing update from remote peer cbf5887cb00342e19f5e051f71bb04a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:44.980619 30350 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:44.980921 30401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:44.981782 30401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:44.990641 30392 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:48:44.992374 30392 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:45.021519 30401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbf5887cb00342e19f5e051f71bb04a2. Latest consensus state: current_term: 1 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:45.021806 30401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:45.021824 30350 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:45.022094 30350 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:45.022750 30410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:45.023120 30410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:45.031332 30392 catalog_manager.cc:6092] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Generated new TSK 0
I20260809 07:48:45.032531 30392 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:45.042910 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:45.072286 30419 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:45.072412 30419 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: loading cluster ID for follower catalog manager: success
I20260809 07:48:45.076274 30419 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: acquiring CA information for follower catalog manager: success
I20260809 07:48:45.087107 30419 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:48:45.494804 30420 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:45.495114 30420 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:45.495239 30420 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:45.503861 30420 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:45.504072 30420 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:45.514294 30420 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:45.516021 30420 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:45.518185 30420 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:45.528112 30428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:45.529136 30427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:45.534539 30430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:45.535290 30420 server_base.cc:1061] running on GCE node
I20260809 07:48:45.536065 30420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:45.537276 30420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:45.538563 30420 hybrid_clock.cc:648] HybridClock initialized: now 1786261725538497 us; error 51 us; skew 500 ppm
I20260809 07:48:45.538985 30420 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:45.541970 30420 webserver.cc:533] Webserver started at http://127.26.37.1:43249/ using document root <none> and password file <none>
I20260809 07:48:45.542980 30420 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:45.543113 30420 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:45.543514 30420 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:45.546931 30420 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:45.548120 30420 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:45.555799 30420 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260809 07:48:45.560433 30437 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.562465 30420 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:48:45.562644 30420 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:45.562891 30420 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:45.589015 30420 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:45.590044 30420 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:45.590341 30420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:45.591320 30420 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:45.593075 30420 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:45.593195 30420 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.593276 30420 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:45.593328 30420 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.629283 30420 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44669
I20260809 07:48:45.629395 30588 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44669 every 8 connection(s)
I20260809 07:48:45.630950 30420 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:45.632962 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30420
I20260809 07:48:45.633201 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:45.640528 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:45.675981 30591 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:45.676450 30591 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:45.677520 30591 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:45.680208 30296 ts_manager.cc:194] Registered new tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:44669)
I20260809 07:48:45.681979 30595 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42757
I20260809 07:48:45.682137 30595 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:45.682698 30595 heartbeater.cc:507] Master 127.26.37.61:42757 requested a full tablet report, sending...
I20260809 07:48:45.685016 30199 ts_manager.cc:194] Registered new tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:44669)
I20260809 07:48:45.686115 30590 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:45.686246 30590 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:45.686661 30590 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:45.688077 30111 ts_manager.cc:194] Registered new tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:44669)
I20260809 07:48:45.689957 30111 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:54669
I20260809 07:48:45.742491 30387 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:45.742650 30387 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: loading cluster ID for follower catalog manager: success
I20260809 07:48:45.745214 30387 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: acquiring CA information for follower catalog manager: success
I20260809 07:48:45.747166 30387 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:48:46.023334 30604 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:46.023702 30604 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:46.023849 30604 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:46.032011 30604 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:46.032246 30604 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:48:46.042690 30604 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:46.044400 30604 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:46.046459 30604 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:46.056440 30614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:46.056524 30615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:46.057394 30618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:46.058073 30604 server_base.cc:1061] running on GCE node
I20260809 07:48:46.058691 30604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:46.059716 30604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:46.060957 30604 hybrid_clock.cc:648] HybridClock initialized: now 1786261726060897 us; error 47 us; skew 500 ppm
I20260809 07:48:46.061264 30604 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:46.064049 30604 webserver.cc:533] Webserver started at http://127.26.37.2:34665/ using document root <none> and password file <none>
I20260809 07:48:46.064945 30604 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:46.065104 30604 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:46.065560 30604 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:46.068130 30604 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "e634609e6b2347198ac3ad7b4ac22823"
format_stamp: "Formatted at 2026-08-09 07:48:46 on dist-test-slave-bw88"
I20260809 07:48:46.069038 30604 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "e634609e6b2347198ac3ad7b4ac22823"
format_stamp: "Formatted at 2026-08-09 07:48:46 on dist-test-slave-bw88"
I20260809 07:48:46.074630 30604 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:48:46.078344 30627 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:46.079684 30604 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:48:46.079875 30604 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "e634609e6b2347198ac3ad7b4ac22823"
format_stamp: "Formatted at 2026-08-09 07:48:46 on dist-test-slave-bw88"
I20260809 07:48:46.080106 30604 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:46.094556 30604 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:46.095409 30604 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:46.095696 30604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:46.096645 30604 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:46.098416 30604 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:46.098546 30604 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:46.098685 30604 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:46.098774 30604 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:46.135669 30604 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:35923
I20260809 07:48:46.135725 30781 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:35923 every 8 connection(s)
I20260809 07:48:46.137401 30604 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:46.142616 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30604
I20260809 07:48:46.142859 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:46.147516 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:46.157968 30785 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42757
I20260809 07:48:46.158385 30785 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:46.159322 30785 heartbeater.cc:507] Master 127.26.37.61:42757 requested a full tablet report, sending...
I20260809 07:48:46.160211 30782 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:46.160348 30782 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:46.160796 30782 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:46.161115 30784 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:46.161280 30784 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:46.161302 30199 ts_manager.cc:194] Registered new tserver with Master: e634609e6b2347198ac3ad7b4ac22823 (127.26.37.2:35923)
I20260809 07:48:46.161793 30784 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:46.162051 30111 ts_manager.cc:194] Registered new tserver with Master: e634609e6b2347198ac3ad7b4ac22823 (127.26.37.2:35923)
I20260809 07:48:46.162920 30296 ts_manager.cc:194] Registered new tserver with Master: e634609e6b2347198ac3ad7b4ac22823 (127.26.37.2:35923)
I20260809 07:48:46.163303 30111 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:51523
W20260809 07:48:46.502791 30790 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:46.503168 30790 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:46.503309 30790 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:46.511950 30790 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:46.512184 30790 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:48:46.522819 30790 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:46.524578 30790 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:46.526624 30790 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:46.536314 30797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:46.536350 30798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:46.537232 30802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:46.537925 30790 server_base.cc:1061] running on GCE node
I20260809 07:48:46.538524 30790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:46.539449 30790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:46.540701 30790 hybrid_clock.cc:648] HybridClock initialized: now 1786261726540659 us; error 53 us; skew 500 ppm
I20260809 07:48:46.541038 30790 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:46.543334 30790 webserver.cc:533] Webserver started at http://127.26.37.3:34677/ using document root <none> and password file <none>
I20260809 07:48:46.544118 30790 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:46.544268 30790 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:46.544646 30790 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:46.547255 30790 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "67a051d48a384bcf9bf5ac24a0293e3a"
format_stamp: "Formatted at 2026-08-09 07:48:46 on dist-test-slave-bw88"
I20260809 07:48:46.548125 30790 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "67a051d48a384bcf9bf5ac24a0293e3a"
format_stamp: "Formatted at 2026-08-09 07:48:46 on dist-test-slave-bw88"
I20260809 07:48:46.553725 30790 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:48:46.557111 30810 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:46.558534 30790 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:46.558717 30790 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "67a051d48a384bcf9bf5ac24a0293e3a"
format_stamp: "Formatted at 2026-08-09 07:48:46 on dist-test-slave-bw88"
I20260809 07:48:46.558966 30790 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:46.580943 30790 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:46.581908 30790 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:46.582221 30790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:46.583324 30790 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:46.585186 30790 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:46.585319 30790 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:46.585443 30790 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:46.585532 30790 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:46.620270 30790 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:43061
I20260809 07:48:46.620358 30949 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:43061 every 8 connection(s)
I20260809 07:48:46.621941 30790 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:46.624821 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30790
I20260809 07:48:46.625066 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:46.650315 30952 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42757
I20260809 07:48:46.650681 30952 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:46.651713 30952 heartbeater.cc:507] Master 127.26.37.61:42757 requested a full tablet report, sending...
I20260809 07:48:46.652623 30950 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:46.652740 30950 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:46.653206 30950 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:46.653477 30951 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:46.653597 30951 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:46.654081 30951 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:46.654124 30199 ts_manager.cc:194] Registered new tserver with Master: 67a051d48a384bcf9bf5ac24a0293e3a (127.26.37.3:43061)
I20260809 07:48:46.655058 30111 ts_manager.cc:194] Registered new tserver with Master: 67a051d48a384bcf9bf5ac24a0293e3a (127.26.37.3:43061)
I20260809 07:48:46.655370 30294 ts_manager.cc:194] Registered new tserver with Master: 67a051d48a384bcf9bf5ac24a0293e3a (127.26.37.3:43061)
I20260809 07:48:46.656044 30111 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:60071
I20260809 07:48:46.669899 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:48:46.676882 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:48:46.677085 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30063
I20260809 07:48:46.678015 30142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:46.692530 30590 heartbeater.cc:499] Master 127.26.37.62:36285 was elected leader, sending a full tablet report...
I20260809 07:48:46.789543 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30063
I20260809 07:48:46.820767 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36285
--webserver_interface=127.26.37.62
--webserver_port=34945
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025 with env {}
I20260809 07:48:47.165490 30782 heartbeater.cc:499] Master 127.26.37.62:36285 was elected leader, sending a full tablet report...
W20260809 07:48:47.167073 30782 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:36285 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:36285: connect: Connection refused (error 111)
W20260809 07:48:47.170697 30959 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:47.171003 30959 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:47.171072 30959 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:47.179399 30959 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:47.179493 30959 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:47.179551 30959 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:47.179591 30959 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:47.189715 30959 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=34945
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:47.191380 30959 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:47.193718 30959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:47.204622 30972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.204782 30968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.205197 30969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:47.207927 30959 server_base.cc:1061] running on GCE node
I20260809 07:48:47.208797 30959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:47.210510 30959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:47.211768 30959 hybrid_clock.cc:648] HybridClock initialized: now 1786261727211715 us; error 63 us; skew 500 ppm
I20260809 07:48:47.212190 30959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:47.214623 30959 webserver.cc:533] Webserver started at http://127.26.37.62:34945/ using document root <none> and password file <none>
I20260809 07:48:47.215476 30959 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:47.215668 30959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:47.221989 30959 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:48:47.225137 30980 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.226554 30959 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:47.226788 30959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cbf5887cb00342e19f5e051f71bb04a2"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-bw88"
I20260809 07:48:47.227638 30959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:47.255946 30959 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:47.257091 30959 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:47.257503 30959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:47.280251 30959 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36285
I20260809 07:48:47.280275 31045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36285 every 8 connection(s)
I20260809 07:48:47.282243 30959 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:47.289556 31047 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:47.291520 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30959
I20260809 07:48:47.291858 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30420
I20260809 07:48:47.293305 31047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap starting.
I20260809 07:48:47.300902 30583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:47.314270 31047 log.cc:826] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:47.325693 31047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:47.326327 31047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap complete.
I20260809 07:48:47.336280 31047 raft_consensus.cc:359] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:47.337261 31047 raft_consensus.cc:740] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf5887cb00342e19f5e051f71bb04a2, State: Initialized, Role: FOLLOWER
I20260809 07:48:47.338171 31047 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:47.339010 31053 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:47.339288 31053 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:47.340240 31047 sys_catalog.cc:565] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:47.350823 31068 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:47.351055 31068 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: loading cluster ID for follower catalog manager: success
I20260809 07:48:47.355355 31068 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: acquiring CA information for follower catalog manager: success
I20260809 07:48:47.358762 31068 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:47.395879 30959 master_runner.cc:264] No leader master found from master cbf5887cb00342e19f5e051f71bb04a2
I20260809 07:48:47.396081 30959 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:47.414280 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30420
I20260809 07:48:47.436239 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44669
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--webserver_port=43249
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:47.658401 30950 heartbeater.cc:499] Master 127.26.37.62:36285 was elected leader, sending a full tablet report...
I20260809 07:48:47.675203 30999 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "67a051d48a384bcf9bf5ac24a0293e3a" instance_seqno: 1786261726611957) as {username='slave'} at 127.26.37.3:58895; Asking this server to re-register.
I20260809 07:48:47.676215 30950 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:47.678088 30999 ts_manager.cc:194] Registered new tserver with Master: 67a051d48a384bcf9bf5ac24a0293e3a (127.26.37.3:43061)
W20260809 07:48:47.797955 31078 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:47.798314 31078 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:47.798455 31078 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:47.806560 31078 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:47.806803 31078 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:47.816289 31078 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44669
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=43249
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:47.818210 31078 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:47.820010 31078 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:47.829981 31088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.830014 31086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.830557 31090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:47.831471 31078 server_base.cc:1061] running on GCE node
I20260809 07:48:47.832069 31078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:47.833125 31078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:47.834369 31078 hybrid_clock.cc:648] HybridClock initialized: now 1786261727834332 us; error 51 us; skew 500 ppm
I20260809 07:48:47.834694 31078 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:47.835381 31093 raft_consensus.cc:493] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cbf5887cb00342e19f5e051f71bb04a2)
I20260809 07:48:47.835687 31093 raft_consensus.cc:515] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:47.836884 31078 webserver.cc:533] Webserver started at http://127.26.37.1:43249/ using document root <none> and password file <none>
I20260809 07:48:47.837278 31093 leader_election.cc:290] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cbf5887cb00342e19f5e051f71bb04a2 (127.26.37.62:36285), 5a028f1f75b94a56adade49eb2e6d92d (127.26.37.60:35025)
I20260809 07:48:47.837996 31078 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:47.838150 31078 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:47.838029 30312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "950ff7120dbc4fbebd2065da8b3adc57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5a028f1f75b94a56adade49eb2e6d92d" is_pre_election: true
I20260809 07:48:47.844682 31078 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:48:47.844852 31012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "950ff7120dbc4fbebd2065da8b3adc57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cbf5887cb00342e19f5e051f71bb04a2" is_pre_election: true
I20260809 07:48:47.845435 31012 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 950ff7120dbc4fbebd2065da8b3adc57 in term 1.
I20260809 07:48:47.846071 30180 leader_election.cc:304] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 950ff7120dbc4fbebd2065da8b3adc57, cbf5887cb00342e19f5e051f71bb04a2; no voters: 5a028f1f75b94a56adade49eb2e6d92d
I20260809 07:48:47.846470 31093 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:48:47.846612 31093 raft_consensus.cc:493] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cbf5887cb00342e19f5e051f71bb04a2)
I20260809 07:48:47.846721 31093 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:47.847949 31099 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.849422 31078 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:47.849553 31093 raft_consensus.cc:515] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:47.850175 31093 leader_election.cc:290] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [CANDIDATE]: Term 2 election: Requested vote from peers cbf5887cb00342e19f5e051f71bb04a2 (127.26.37.62:36285), 5a028f1f75b94a56adade49eb2e6d92d (127.26.37.60:35025)
I20260809 07:48:47.849566 31078 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:47.850353 31078 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:47.850513 31012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "950ff7120dbc4fbebd2065da8b3adc57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cbf5887cb00342e19f5e051f71bb04a2"
I20260809 07:48:47.850747 31012 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:47.850761 30312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "950ff7120dbc4fbebd2065da8b3adc57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5a028f1f75b94a56adade49eb2e6d92d"
I20260809 07:48:47.854328 31012 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 950ff7120dbc4fbebd2065da8b3adc57 in term 2.
I20260809 07:48:47.854790 30180 leader_election.cc:304] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 950ff7120dbc4fbebd2065da8b3adc57, cbf5887cb00342e19f5e051f71bb04a2; no voters: 5a028f1f75b94a56adade49eb2e6d92d
I20260809 07:48:47.855079 31093 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:48:47.855578 31093 raft_consensus.cc:697] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 2 LEADER]: Becoming Leader. State: Replica: 950ff7120dbc4fbebd2065da8b3adc57, State: Running, Role: LEADER
I20260809 07:48:47.856001 31093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:47.858090 31100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 950ff7120dbc4fbebd2065da8b3adc57. Latest consensus state: current_term: 2 leader_uuid: "950ff7120dbc4fbebd2065da8b3adc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:47.858451 31100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:47.859151 31107 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:47.861037 31107 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:47.861938 31107 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:47.862010 31107 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:47.863169 31107 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:47.864037 31107 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: Loaded TSK: 0
I20260809 07:48:47.864857 31107 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:47.883427 31078 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:47.884334 31078 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:47.884660 31078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:47.885756 31078 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:47.887449 31078 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:47.887562 31078 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.887723 31078 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:47.887806 31078 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.922962 31078 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44669
I20260809 07:48:47.923058 31242 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44669 every 8 connection(s)
I20260809 07:48:47.924504 31078 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:47.930706 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31078
I20260809 07:48:47.931008 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31078
I20260809 07:48:47.951895 31245 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42757
I20260809 07:48:47.952354 31245 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:47.952682 31243 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:47.952803 31243 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:47.953054 31238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:48.038116 31244 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:48.038489 31244 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:48.038496 31243 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:48.038759 31245 heartbeater.cc:507] Master 127.26.37.61:42757 requested a full tablet report, sending...
I20260809 07:48:48.040156 31244 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:48.040374 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31078
I20260809 07:48:48.041002 30198 ts_manager.cc:194] Re-registered known tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:44669)
I20260809 07:48:48.042850 30198 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:55729
I20260809 07:48:48.044477 30999 ts_manager.cc:194] Registered new tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:44669)
W20260809 07:48:48.065834 30180 connection.cc:573] server connection from 127.26.37.1:55729 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:48:48.065896 30983 connection.cc:573] server connection from 127.26.37.1:59033 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:48:48.066849 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44669
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--webserver_port=43249
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260809 07:48:48.069401 31259 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cbf5887cb00342e19f5e051f71bb04a2)
I20260809 07:48:48.069675 31259 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:48.071105 31259 leader_election.cc:290] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cbf5887cb00342e19f5e051f71bb04a2 (127.26.37.62:36285), 950ff7120dbc4fbebd2065da8b3adc57 (127.26.37.61:42757)
I20260809 07:48:48.071854 30208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a028f1f75b94a56adade49eb2e6d92d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "950ff7120dbc4fbebd2065da8b3adc57" is_pre_election: true
I20260809 07:48:48.078519 31012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a028f1f75b94a56adade49eb2e6d92d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cbf5887cb00342e19f5e051f71bb04a2" is_pre_election: true
I20260809 07:48:48.078959 31012 raft_consensus.cc:2393] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a028f1f75b94a56adade49eb2e6d92d in current term 2: Already voted for candidate 950ff7120dbc4fbebd2065da8b3adc57 in this term.
I20260809 07:48:48.079741 30271 leader_election.cc:304] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a028f1f75b94a56adade49eb2e6d92d; no voters: 950ff7120dbc4fbebd2065da8b3adc57, cbf5887cb00342e19f5e051f71bb04a2
I20260809 07:48:48.080139 31259 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:48.082451 31259 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:48:48.175693 30782 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:48.175948 30782 heartbeater.cc:499] Master 127.26.37.62:36285 was elected leader, sending a full tablet report...
I20260809 07:48:48.177132 30999 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e634609e6b2347198ac3ad7b4ac22823" instance_seqno: 1786261726126682) as {username='slave'} at 127.26.37.2:44845; Asking this server to re-register.
I20260809 07:48:48.177937 30782 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:48.179227 30999 ts_manager.cc:194] Registered new tserver with Master: e634609e6b2347198ac3ad7b4ac22823 (127.26.37.2:35923)
I20260809 07:48:48.238415 31012 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Refusing update from remote peer 950ff7120dbc4fbebd2065da8b3adc57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:48:48.239300 31100 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:48:48.243199 31263 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 950ff7120dbc4fbebd2065da8b3adc57. Latest consensus state: current_term: 2 leader_uuid: "950ff7120dbc4fbebd2065da8b3adc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:48.243533 31263 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:48.245088 31093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "950ff7120dbc4fbebd2065da8b3adc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:48.245555 31093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:48.246251 30312 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 2 FOLLOWER]: Refusing update from remote peer 950ff7120dbc4fbebd2065da8b3adc57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:48:48.247087 31093 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:48:48.247565 31263 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "950ff7120dbc4fbebd2065da8b3adc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:48.247802 31263 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:48.264040 31259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 950ff7120dbc4fbebd2065da8b3adc57. Latest consensus state: current_term: 2 leader_uuid: "950ff7120dbc4fbebd2065da8b3adc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:48.264333 31259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:48.263917 31269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "950ff7120dbc4fbebd2065da8b3adc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:48.264537 31269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:48:48.425143 31261 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:48.425582 31261 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:48.425727 31261 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:48:48.425819 31261 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:48.434353 31261 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:48.434636 31261 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:48.444660 31261 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44669
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=43249
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:48.446436 31261 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:48.448367 31261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:48.458094 31280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.458102 31275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.458335 31282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.459131 31261 server_base.cc:1061] running on GCE node
I20260809 07:48:48.459704 31261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:48.460675 31261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:48.461959 31261 hybrid_clock.cc:648] HybridClock initialized: now 1786261728461877 us; error 41 us; skew 500 ppm
I20260809 07:48:48.462270 31261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:48.464443 31261 webserver.cc:533] Webserver started at http://127.26.37.1:43249/ using document root <none> and password file <none>
I20260809 07:48:48.465238 31261 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:48.465432 31261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.470579 31261 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 07:48:48.473112 31289 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.474375 31261 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:48.474552 31261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:48.475297 31261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.496057 31261 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:48.496881 31261 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:48.497179 31261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:48.498260 31261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:48.499955 31261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:48.500083 31261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.500223 31261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:48.500312 31261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:48:48.500687 31261 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:48:48.500793 31261 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:48:48.500864 31261 fault_injection.cc:59] injecting fault for kudu (pid 31261): FLAGS_fault_before_start (process will exit)
I20260809 07:48:48.526031 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44669
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--webserver_port=43249
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:48.855921 31301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "950ff7120dbc4fbebd2065da8b3adc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:48.856264 31301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: LEADER
W20260809 07:48:48.868638 31300 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:48.868927 31300 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:48.869114 31300 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:48.877234 31300 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:48.877460 31300 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:48.886999 31300 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44669
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=43249
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:48.888650 31300 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:48.890503 31300 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:48.900409 31312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.900512 31313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.901093 31315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.901816 31300 server_base.cc:1061] running on GCE node
I20260809 07:48:48.902464 31300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:48.903667 31300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:48.904860 31300 hybrid_clock.cc:648] HybridClock initialized: now 1786261728904803 us; error 58 us; skew 500 ppm
I20260809 07:48:48.905251 31300 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:48.907985 31300 webserver.cc:533] Webserver started at http://127.26.37.1:43249/ using document root <none> and password file <none>
I20260809 07:48:48.908951 31300 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:48.909106 31300 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.916533 31300 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 07:48:48.919510 31321 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.920938 31300 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:48.921078 31300 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:48.921957 31300 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.937502 31300 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:48.938306 31300 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:48.938629 31300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:48.939543 31300 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:48.941304 31300 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:48.941524 31300 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 07:48:48.941690 31300 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:48.941788 31300 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.977985 31300 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44669
I20260809 07:48:48.978031 31464 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44669 every 8 connection(s)
I20260809 07:48:48.979539 31300 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:48.986183 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31300
I20260809 07:48:48.986585 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31300
I20260809 07:48:49.025928 31465 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:49.026387 31466 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:49.026479 31465 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:49.026587 31466 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:49.027844 31465 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:49.027858 31466 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:49.027868 31459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:49.120482 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31300
I20260809 07:48:49.120513 30294 ts_manager.cc:194] Re-registered known tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:44669)
I20260809 07:48:49.120888 30999 ts_manager.cc:194] Re-registered known tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:44669)
W20260809 07:48:49.145370 30271 connection.cc:573] server connection from 127.26.37.1:41761 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:48:49.145454 30985 connection.cc:573] server connection from 127.26.37.1:58155 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:48:49.145316 30183 connection.cc:573] server connection from 127.26.37.1:33035 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:48:49.145845 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30604
I20260809 07:48:49.154438 30774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:49.242871 30785 heartbeater.cc:499] Master 127.26.37.61:42757 was elected leader, sending a full tablet report...
I20260809 07:48:49.245262 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30604
I20260809 07:48:49.269484 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30790
I20260809 07:48:49.278255 30944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:49.366457 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30790
I20260809 07:48:49.392802 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30959
I20260809 07:48:49.399214 31040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:49.506292 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30959
I20260809 07:48:49.535712 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30156
I20260809 07:48:49.536980 30237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:49.646935 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30156
I20260809 07:48:49.681901 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30248
I20260809 07:48:49.683100 30334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:49.792275 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30248
2026-08-09T07:48:49Z chronyd exiting
2026-08-09T07:48:49Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:49Z Disabled control of system clock
I20260809 07:48:49.869515 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36285
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025 with env {}
W20260809 07:48:50.224928 31498 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:50.225284 31498 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:50.225445 31498 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:50.233667 31498 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:50.233755 31498 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:50.233863 31498 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:50.233954 31498 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:50.243992 31498 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:50.245754 31498 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:50.247629 31498 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:50.255756 31507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:50.255820 31506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:50.256470 31511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:50.257690 31498 server_base.cc:1061] running on GCE node
I20260809 07:48:50.258456 31498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:50.259812 31498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:50.261051 31498 hybrid_clock.cc:648] HybridClock initialized: now 1786261730261017 us; error 52 us; skew 500 ppm
I20260809 07:48:50.261511 31498 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:50.263829 31498 webserver.cc:533] Webserver started at http://127.26.37.62:45711/ using document root <none> and password file <none>
I20260809 07:48:50.264659 31498 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:50.264846 31498 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:50.270632 31498 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:48:50.273869 31519 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:50.275297 31498 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:48:50.275514 31498 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cbf5887cb00342e19f5e051f71bb04a2"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-bw88"
I20260809 07:48:50.276264 31498 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:50.318629 31498 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:50.319630 31498 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:50.319983 31498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:50.340889 31498 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36285
I20260809 07:48:50.340914 31584 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36285 every 8 connection(s)
I20260809 07:48:50.342693 31498 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:50.349673 31585 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:50.350782 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31498
I20260809 07:48:50.350955 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:48:50.352267 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:42757
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025 with env {}
I20260809 07:48:50.354278 31585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap starting.
I20260809 07:48:50.372517 31585 log.cc:826] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:50.381508 31585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:50.382164 31585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap complete.
I20260809 07:48:50.389215 31585 raft_consensus.cc:359] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:50.389977 31585 raft_consensus.cc:740] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf5887cb00342e19f5e051f71bb04a2, State: Initialized, Role: FOLLOWER
I20260809 07:48:50.390769 31585 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:50.391811 31595 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:50.392122 31595 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:50.392885 31585 sys_catalog.cc:565] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:50.403947 31609 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:50.404060 31609 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: loading cluster ID for follower catalog manager: success
I20260809 07:48:50.410882 31609 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: acquiring CA information for follower catalog manager: success
I20260809 07:48:50.413700 31609 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:50.427429 31498 master_runner.cc:187] Error getting master registration for 127.26.37.61:42757: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42757: connect: Connection refused (error 111),
I20260809 07:48:50.433557 31498 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:48:50.713327 31592 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:50.713784 31592 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:50.713910 31592 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:50.722394 31592 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:50.722502 31592 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:50.722591 31592 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:50.722661 31592 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:50.733220 31592 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:42757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:50.735179 31592 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:50.737133 31592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:50.745361 31620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:50.745838 31621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:50.747409 31623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:50.748137 31592 server_base.cc:1061] running on GCE node
I20260809 07:48:50.748874 31592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:50.750375 31592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:50.751612 31592 hybrid_clock.cc:648] HybridClock initialized: now 1786261730751558 us; error 48 us; skew 500 ppm
I20260809 07:48:50.751999 31592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:50.754321 31592 webserver.cc:533] Webserver started at http://127.26.37.61:36265/ using document root <none> and password file <none>
I20260809 07:48:50.755179 31592 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:50.755339 31592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:50.760953 31592 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 07:48:50.764114 31631 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:50.765676 31592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:48:50.765858 31592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "950ff7120dbc4fbebd2065da8b3adc57"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-bw88"
I20260809 07:48:50.766631 31592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:50.785306 31592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:50.786252 31592 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:50.786578 31592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:50.809985 31592 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:42757
I20260809 07:48:50.810010 31700 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:42757 every 8 connection(s)
I20260809 07:48:50.811677 31592 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:48:50.816442 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31592
I20260809 07:48:50.816673 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:48:50.817996 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:35025
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025 with env {}
I20260809 07:48:50.819904 31701 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:50.825060 31701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: Bootstrap starting.
I20260809 07:48:50.842795 31701 log.cc:826] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:50.851780 31701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:50.852337 31701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: Bootstrap complete.
I20260809 07:48:50.859489 31701 raft_consensus.cc:359] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:50.860193 31701 raft_consensus.cc:740] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950ff7120dbc4fbebd2065da8b3adc57, State: Initialized, Role: FOLLOWER
I20260809 07:48:50.860963 31701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:50.862003 31706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:50.862318 31706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:50.863101 31701 sys_catalog.cc:565] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:50.874137 31719 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:50.874253 31719 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: loading cluster ID for follower catalog manager: success
I20260809 07:48:50.878376 31719 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: acquiring CA information for follower catalog manager: success
I20260809 07:48:50.880810 31719 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:50.908188 31592 master_runner.cc:187] Error getting master registration for 127.26.37.60:35025: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35025: connect: Connection refused (error 111),
I20260809 07:48:50.913988 31592 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:48:51.179147 31703 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:51.179517 31703 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:51.179648 31703 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:51.187947 31703 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:51.188092 31703 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:51.188249 31703 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:51.188328 31703 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:51.198302 31703 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:35025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:51.200001 31703 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:51.202023 31703 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:51.210381 31732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:51.210563 31734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:51.211760 31736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:51.212656 31703 server_base.cc:1061] running on GCE node
I20260809 07:48:51.213430 31703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:51.214815 31703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:51.216045 31703 hybrid_clock.cc:648] HybridClock initialized: now 1786261731215991 us; error 44 us; skew 500 ppm
I20260809 07:48:51.216390 31703 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:51.218776 31703 webserver.cc:533] Webserver started at http://127.26.37.60:36007/ using document root <none> and password file <none>
I20260809 07:48:51.219511 31703 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:51.219661 31703 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:51.225191 31703 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:48:51.228149 31743 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:51.229735 31703 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:51.229905 31703 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "5a028f1f75b94a56adade49eb2e6d92d"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-bw88"
I20260809 07:48:51.230690 31703 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:51.248643 31703 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:51.249522 31703 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:51.249822 31703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:51.271555 31703 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:35025
I20260809 07:48:51.271648 31815 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:35025 every 8 connection(s)
I20260809 07:48:51.273268 31703 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:48:51.280323 31816 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:51.281245 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31703
I20260809 07:48:51.281467 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:48:51.284394 31816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Bootstrap starting.
I20260809 07:48:51.304582 31816 log.cc:826] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:51.314067 31816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:51.314610 31816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Bootstrap complete.
I20260809 07:48:51.320801 31816 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:51.321568 31816 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a028f1f75b94a56adade49eb2e6d92d, State: Initialized, Role: FOLLOWER
I20260809 07:48:51.322232 31816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:51.323566 31822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:51.323844 31822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:51.324584 31816 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:51.334882 31834 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:51.335060 31834 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: loading cluster ID for follower catalog manager: success
I20260809 07:48:51.339886 31834 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: acquiring CA information for follower catalog manager: success
I20260809 07:48:51.342500 31834 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:51.364528 31703 master_runner.cc:264] No leader master found from master 5a028f1f75b94a56adade49eb2e6d92d
I20260809 07:48:51.364712 31703 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:52.107568 31839 raft_consensus.cc:493] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:52.107832 31839 raft_consensus.cc:515] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:52.109598 31839 leader_election.cc:290] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 950ff7120dbc4fbebd2065da8b3adc57 (127.26.37.61:42757), 5a028f1f75b94a56adade49eb2e6d92d (127.26.37.60:35025)
I20260809 07:48:52.118470 31672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbf5887cb00342e19f5e051f71bb04a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "950ff7120dbc4fbebd2065da8b3adc57" is_pre_election: true
I20260809 07:48:52.118557 31780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbf5887cb00342e19f5e051f71bb04a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5a028f1f75b94a56adade49eb2e6d92d" is_pre_election: true
I20260809 07:48:52.119102 31672 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf5887cb00342e19f5e051f71bb04a2 in term 2.
I20260809 07:48:52.119102 31780 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf5887cb00342e19f5e051f71bb04a2 in term 2.
I20260809 07:48:52.119716 31521 leader_election.cc:304] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 950ff7120dbc4fbebd2065da8b3adc57, cbf5887cb00342e19f5e051f71bb04a2; no voters:
I20260809 07:48:52.120102 31839 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:48:52.120265 31839 raft_consensus.cc:493] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:52.120371 31839 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:48:52.124171 31839 raft_consensus.cc:515] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:52.124754 31839 leader_election.cc:290] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [CANDIDATE]: Term 3 election: Requested vote from peers 950ff7120dbc4fbebd2065da8b3adc57 (127.26.37.61:42757), 5a028f1f75b94a56adade49eb2e6d92d (127.26.37.60:35025)
I20260809 07:48:52.125267 31672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbf5887cb00342e19f5e051f71bb04a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "950ff7120dbc4fbebd2065da8b3adc57"
I20260809 07:48:52.125384 31780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cbf5887cb00342e19f5e051f71bb04a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5a028f1f75b94a56adade49eb2e6d92d"
I20260809 07:48:52.125506 31672 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:48:52.125576 31780 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:48:52.130882 31672 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf5887cb00342e19f5e051f71bb04a2 in term 3.
I20260809 07:48:52.130882 31780 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf5887cb00342e19f5e051f71bb04a2 in term 3.
I20260809 07:48:52.131311 31521 leader_election.cc:304] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 950ff7120dbc4fbebd2065da8b3adc57, cbf5887cb00342e19f5e051f71bb04a2; no voters:
I20260809 07:48:52.131603 31839 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:48:52.132148 31839 raft_consensus.cc:697] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 3 LEADER]: Becoming Leader. State: Replica: cbf5887cb00342e19f5e051f71bb04a2, State: Running, Role: LEADER
I20260809 07:48:52.132730 31839 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:52.136487 31846 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbf5887cb00342e19f5e051f71bb04a2. Latest consensus state: current_term: 3 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:52.136831 31846 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:52.137578 31849 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:52.139729 31849 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:52.140551 31849 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:52.140625 31849 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:52.141876 31849 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:52.142675 31849 catalog_manager.cc:6103] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Loaded TSK: 0
I20260809 07:48:52.143571 31849 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:52.179173 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:52.524152 31672 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 3 FOLLOWER]: Refusing update from remote peer cbf5887cb00342e19f5e051f71bb04a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:48:52.525359 31846 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:48:52.529832 31856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbf5887cb00342e19f5e051f71bb04a2. Latest consensus state: current_term: 3 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:52.530125 31856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:52.531121 31846 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:52.531433 31846 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:52.533192 31780 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 3 FOLLOWER]: Refusing update from remote peer cbf5887cb00342e19f5e051f71bb04a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260809 07:48:52.533545 31850 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:52.534125 31850 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:52.534337 31850 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:48:52.534302 31846 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:48:52.546988 31856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:52.547253 31856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:52.550366 31862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbf5887cb00342e19f5e051f71bb04a2. Latest consensus state: current_term: 3 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:52.550632 31862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:52.550859 31862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:52.550993 31862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:48:52.550971 31850 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:52.551136 31850 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:52.561446 31850 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:52.563059 31850 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:52.564884 31850 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:52.575346 31868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:52.575276 31869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:52.576351 31871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:52.577196 31850 server_base.cc:1061] running on GCE node
I20260809 07:48:52.577826 31850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:52.578727 31850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:52.579958 31850 hybrid_clock.cc:648] HybridClock initialized: now 1786261732579918 us; error 70 us; skew 500 ppm
I20260809 07:48:52.580283 31850 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:52.582556 31850 webserver.cc:533] Webserver started at http://127.26.37.1:43205/ using document root <none> and password file <none>
I20260809 07:48:52.583364 31850 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:52.583515 31850 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:52.589330 31850 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260809 07:48:52.592387 31879 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:52.594177 31850 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:52.594420 31850 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:52.595268 31850 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:52.614322 31850 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:52.615226 31850 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:52.615525 31850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:52.616511 31850 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:52.618541 31850 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:52.618651 31850 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:52.618795 31850 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:52.618876 31850 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:52.654433 31850 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:40917
I20260809 07:48:52.654471 32023 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:40917 every 8 connection(s)
I20260809 07:48:52.655998 31850 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:52.661669 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31850
I20260809 07:48:52.661935 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:52.668913 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:52.703428 32024 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:52.703804 32026 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42757
I20260809 07:48:52.703935 32024 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:52.703981 32026 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:52.704609 32025 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:52.704762 32025 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:52.705417 32024 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:52.705482 32026 heartbeater.cc:507] Master 127.26.37.61:42757 requested a full tablet report, sending...
I20260809 07:48:52.705487 32025 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:52.708117 31541 ts_manager.cc:194] Registered new tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:40917)
I20260809 07:48:52.708149 31654 ts_manager.cc:194] Registered new tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:40917)
I20260809 07:48:52.708304 31769 ts_manager.cc:194] Registered new tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:40917)
I20260809 07:48:52.710105 31541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:34387
W20260809 07:48:53.041442 32034 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:53.041779 32034 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:53.041868 32034 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:48:53.045758 31839 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cbf5887cb00342e19f5e051f71bb04a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:53.046063 31839 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: LEADER
W20260809 07:48:53.051059 32034 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:53.051285 32034 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:48:53.061544 32034 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:53.063234 32034 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:53.065110 32034 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:53.074681 32042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:53.074846 32043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:53.075312 32045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:53.076225 32034 server_base.cc:1061] running on GCE node
I20260809 07:48:53.076887 32034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:53.077914 32034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:53.079180 32034 hybrid_clock.cc:648] HybridClock initialized: now 1786261733079111 us; error 56 us; skew 500 ppm
I20260809 07:48:53.079532 32034 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:53.081964 32034 webserver.cc:533] Webserver started at http://127.26.37.2:32905/ using document root <none> and password file <none>
I20260809 07:48:53.082814 32034 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:53.082978 32034 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:53.091217 32034 fs_manager.cc:714] Time spent opening directory manager: real 0.006s user 0.003s sys 0.002s
I20260809 07:48:53.094156 32052 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:53.095513 32034 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:53.095686 32034 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "e634609e6b2347198ac3ad7b4ac22823"
format_stamp: "Formatted at 2026-08-09 07:48:46 on dist-test-slave-bw88"
I20260809 07:48:53.096422 32034 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:53.120105 32034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:53.120929 32034 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:53.121264 32034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:53.122287 32034 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:53.123935 32034 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:53.124040 32034 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:53.124159 32034 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:53.124228 32034 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:53.159969 32034 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:46637
I20260809 07:48:53.160054 32192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:46637 every 8 connection(s)
I20260809 07:48:53.161599 32034 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:53.162575 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32034
I20260809 07:48:53.162813 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:53.169835 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:53.191174 32195 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:53.191287 32196 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42757
I20260809 07:48:53.191347 32194 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:53.191665 32194 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:53.191661 32196 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:53.191663 32195 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:53.193001 32195 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:53.193040 32194 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:53.193066 32196 heartbeater.cc:507] Master 127.26.37.61:42757 requested a full tablet report, sending...
I20260809 07:48:53.195003 31541 ts_manager.cc:194] Registered new tserver with Master: e634609e6b2347198ac3ad7b4ac22823 (127.26.37.2:46637)
I20260809 07:48:53.195036 31769 ts_manager.cc:194] Registered new tserver with Master: e634609e6b2347198ac3ad7b4ac22823 (127.26.37.2:46637)
I20260809 07:48:53.195958 31541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:41175
I20260809 07:48:53.196730 31654 ts_manager.cc:194] Registered new tserver with Master: e634609e6b2347198ac3ad7b4ac22823 (127.26.37.2:46637)
W20260809 07:48:53.531543 32202 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:53.531821 32202 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:53.531894 32202 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:53.540119 32202 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:53.540329 32202 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:48:53.550788 32202 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:53.552450 32202 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:53.554382 32202 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:53.563861 32208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:53.564491 32207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:53.565364 32210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:53.565742 32202 server_base.cc:1061] running on GCE node
I20260809 07:48:53.566356 32202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:53.567337 32202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:53.568620 32202 hybrid_clock.cc:648] HybridClock initialized: now 1786261733568563 us; error 45 us; skew 500 ppm
I20260809 07:48:53.568936 32202 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:53.571194 32202 webserver.cc:533] Webserver started at http://127.26.37.3:34859/ using document root <none> and password file <none>
I20260809 07:48:53.572036 32202 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:53.572207 32202 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:53.577894 32202 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:48:53.580632 32221 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:53.582074 32202 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:53.582250 32202 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "67a051d48a384bcf9bf5ac24a0293e3a"
format_stamp: "Formatted at 2026-08-09 07:48:46 on dist-test-slave-bw88"
I20260809 07:48:53.583027 32202 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:53.605312 32202 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:53.606432 32202 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:53.606762 32202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:53.607827 32202 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:53.609810 32202 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:53.609890 32202 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:53.610042 32202 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:53.610116 32202 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:53.646673 32202 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:34749
I20260809 07:48:53.646715 32363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:34749 every 8 connection(s)
I20260809 07:48:53.648263 32202 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:53.650360 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32202
I20260809 07:48:53.650580 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:53.668855 32365 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:53.669289 32365 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:53.670495 32365 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:53.672546 31769 ts_manager.cc:194] Registered new tserver with Master: 67a051d48a384bcf9bf5ac24a0293e3a (127.26.37.3:34749)
I20260809 07:48:53.673249 32364 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:53.673409 32364 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:53.673913 32364 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:53.674784 31541 ts_manager.cc:194] Registered new tserver with Master: 67a051d48a384bcf9bf5ac24a0293e3a (127.26.37.3:34749)
I20260809 07:48:53.675098 32366 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42757
I20260809 07:48:53.675243 32366 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:53.675648 32366 heartbeater.cc:507] Master 127.26.37.61:42757 requested a full tablet report, sending...
I20260809 07:48:53.675686 31541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:39389
I20260809 07:48:53.676743 31654 ts_manager.cc:194] Registered new tserver with Master: 67a051d48a384bcf9bf5ac24a0293e3a (127.26.37.3:34749)
I20260809 07:48:53.683797 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:48:53.691360 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:48:53.691558 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31498
I20260809 07:48:53.692385 31579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:53.712569 32024 heartbeater.cc:499] Master 127.26.37.62:36285 was elected leader, sending a full tablet report...
I20260809 07:48:53.803112 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31498
I20260809 07:48:53.835116 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36285
--webserver_interface=127.26.37.62
--webserver_port=45711
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025 with env {}
W20260809 07:48:54.184465 32374 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:54.184927 32374 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:54.185052 32374 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:54.193655 32374 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:54.193737 32374 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:54.193825 32374 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:54.193928 32374 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:54.199167 32194 heartbeater.cc:499] Master 127.26.37.62:36285 was elected leader, sending a full tablet report...
W20260809 07:48:54.200984 32194 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:36285 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:36285: connect: Connection refused (error 111)
I20260809 07:48:54.205169 32374 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=45711
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:54.206786 32374 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:54.208679 32374 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:54.216576 32381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:54.217392 32380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:54.217482 32385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:54.218055 32374 server_base.cc:1061] running on GCE node
I20260809 07:48:54.218817 32374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:54.220196 32374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:54.221444 32374 hybrid_clock.cc:648] HybridClock initialized: now 1786261734221405 us; error 62 us; skew 500 ppm
I20260809 07:48:54.221794 32374 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:54.223901 32374 webserver.cc:533] Webserver started at http://127.26.37.62:45711/ using document root <none> and password file <none>
I20260809 07:48:54.224625 32374 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:54.224767 32374 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:54.230259 32374 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:48:54.235869 32392 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:54.237301 32374 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 07:48:54.237543 32374 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cbf5887cb00342e19f5e051f71bb04a2"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-bw88"
I20260809 07:48:54.238243 32374 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:54.252907 32374 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:54.253760 32374 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:54.254011 32374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:54.275727 32374 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36285
I20260809 07:48:54.275820 32462 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36285 every 8 connection(s)
I20260809 07:48:54.277590 32374 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:54.284801 32463 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:54.286516 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32374
I20260809 07:48:54.286845 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31850
I20260809 07:48:54.288725 32463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap starting.
I20260809 07:48:54.296804 32017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:54.307866 32463 log.cc:826] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:54.318172 32463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:54.318785 32463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: Bootstrap complete.
I20260809 07:48:54.325896 32463 raft_consensus.cc:359] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:54.326682 32463 raft_consensus.cc:740] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf5887cb00342e19f5e051f71bb04a2, State: Initialized, Role: FOLLOWER
I20260809 07:48:54.327528 32463 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:54.328606 32470 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:54.328979 32470 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:54.329927 32463 sys_catalog.cc:565] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:54.352627 32482 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:54.352790 32482 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: loading cluster ID for follower catalog manager: success
I20260809 07:48:54.356904 32482 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: acquiring CA information for follower catalog manager: success
I20260809 07:48:54.359539 32482 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:54.381760 32374 master_runner.cc:264] No leader master found from master cbf5887cb00342e19f5e051f71bb04a2
I20260809 07:48:54.381951 32374 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:54.403756 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31850
I20260809 07:48:54.428296 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:40917
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--webserver_port=43205
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:54.678098 32364 heartbeater.cc:499] Master 127.26.37.62:36285 was elected leader, sending a full tablet report...
I20260809 07:48:54.690091 32419 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "67a051d48a384bcf9bf5ac24a0293e3a" instance_seqno: 1786261733638143) as {username='slave'} at 127.26.37.3:41969; Asking this server to re-register.
I20260809 07:48:54.691128 32364 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:54.693173 32419 ts_manager.cc:194] Registered new tserver with Master: 67a051d48a384bcf9bf5ac24a0293e3a (127.26.37.3:34749)
W20260809 07:48:54.787433 32492 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:54.787811 32492 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:54.787959 32492 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:54.797132 32492 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:54.797429 32492 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:54.807443 32492 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:40917
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=43205
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:54.809224 32492 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:54.811640 32492 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:54.822952 32502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:54.823390 32501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:54.824118 32504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:54.825011 32492 server_base.cc:1061] running on GCE node
I20260809 07:48:54.825618 32492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:54.826498 32492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:54.827721 32492 hybrid_clock.cc:648] HybridClock initialized: now 1786261734827687 us; error 30 us; skew 500 ppm
I20260809 07:48:54.828058 32492 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:54.830451 32492 webserver.cc:533] Webserver started at http://127.26.37.1:43205/ using document root <none> and password file <none>
I20260809 07:48:54.831284 32492 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:54.831394 32492 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:54.837143 32492 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:48:54.840024 32511 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:54.841516 32492 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:54.841686 32492 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:54.842427 32492 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:54.868338 32492 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:54.869232 32492 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:54.869560 32492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:54.870749 32492 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:54.872546 32492 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:54.872619 32492 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:54.872802 32492 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:54.872861 32492 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:54.909974 32492 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:40917
I20260809 07:48:54.910068 32650 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:40917 every 8 connection(s)
I20260809 07:48:54.911643 32492 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:54.922142 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32492
I20260809 07:48:54.922472 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32492
I20260809 07:48:54.931028 32654 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42757
I20260809 07:48:54.931514 32654 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:54.932648 32654 heartbeater.cc:507] Master 127.26.37.61:42757 requested a full tablet report, sending...
I20260809 07:48:54.933458 32652 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:54.933627 32652 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:54.934101 32652 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:54.934470 31654 ts_manager.cc:194] Re-registered known tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:40917)
I20260809 07:48:54.935146 32419 ts_manager.cc:194] Registered new tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:40917)
I20260809 07:48:54.939459 32653 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35025
I20260809 07:48:54.939668 32653 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:54.940194 32653 heartbeater.cc:507] Master 127.26.37.60:35025 requested a full tablet report, sending...
I20260809 07:48:54.940969 32646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:54.941514 31769 ts_manager.cc:194] Re-registered known tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:40917)
I20260809 07:48:54.999716 32665 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cbf5887cb00342e19f5e051f71bb04a2)
I20260809 07:48:54.999979 32665 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:55.002288 32665 leader_election.cc:290] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cbf5887cb00342e19f5e051f71bb04a2 (127.26.37.62:36285), 950ff7120dbc4fbebd2065da8b3adc57 (127.26.37.61:42757)
I20260809 07:48:55.002946 31672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a028f1f75b94a56adade49eb2e6d92d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "950ff7120dbc4fbebd2065da8b3adc57" is_pre_election: true
I20260809 07:48:55.009591 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a028f1f75b94a56adade49eb2e6d92d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cbf5887cb00342e19f5e051f71bb04a2" is_pre_election: true
I20260809 07:48:55.010198 32432 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a028f1f75b94a56adade49eb2e6d92d in term 3.
I20260809 07:48:55.010787 31744 leader_election.cc:304] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a028f1f75b94a56adade49eb2e6d92d, cbf5887cb00342e19f5e051f71bb04a2; no voters: 950ff7120dbc4fbebd2065da8b3adc57
I20260809 07:48:55.011209 32665 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:48:55.011353 32665 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 3 FOLLOWER]: Starting leader election (detected failure of leader cbf5887cb00342e19f5e051f71bb04a2)
I20260809 07:48:55.011478 32665 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:48:55.013720 32665 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:55.014405 32665 leader_election.cc:290] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [CANDIDATE]: Term 4 election: Requested vote from peers cbf5887cb00342e19f5e051f71bb04a2 (127.26.37.62:36285), 950ff7120dbc4fbebd2065da8b3adc57 (127.26.37.61:42757)
I20260809 07:48:55.014820 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a028f1f75b94a56adade49eb2e6d92d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cbf5887cb00342e19f5e051f71bb04a2"
I20260809 07:48:55.015046 32432 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:48:55.015033 31672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a028f1f75b94a56adade49eb2e6d92d" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "950ff7120dbc4fbebd2065da8b3adc57"
I20260809 07:48:55.019351 32432 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a028f1f75b94a56adade49eb2e6d92d in term 4.
I20260809 07:48:55.019804 31744 leader_election.cc:304] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a028f1f75b94a56adade49eb2e6d92d, cbf5887cb00342e19f5e051f71bb04a2; no voters: 950ff7120dbc4fbebd2065da8b3adc57
I20260809 07:48:55.020125 32665 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:48:55.020576 32665 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [term 4 LEADER]: Becoming Leader. State: Replica: 5a028f1f75b94a56adade49eb2e6d92d, State: Running, Role: LEADER
I20260809 07:48:55.021018 32665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:55.022759 32667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a028f1f75b94a56adade49eb2e6d92d. Latest consensus state: current_term: 4 leader_uuid: "5a028f1f75b94a56adade49eb2e6d92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:55.023025 32667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:55.023820 32669 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:48:55.026422 32669 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:55.027426 32669 catalog_manager.cc:1295] Loaded cluster ID: 20cdd57196d64d9b978a5a7e2bae55c8
I20260809 07:48:55.027525 32669 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:55.028996 32669 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:55.030035 32669 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d: Loaded TSK: 0
I20260809 07:48:55.031134 32669 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:55.035245 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32492
I20260809 07:48:55.060830 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:40917
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--webserver_port=43205
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260809 07:48:55.128811 32672 raft_consensus.cc:493] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cbf5887cb00342e19f5e051f71bb04a2)
I20260809 07:48:55.129061 32672 raft_consensus.cc:515] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } }
I20260809 07:48:55.130932 32672 leader_election.cc:290] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cbf5887cb00342e19f5e051f71bb04a2 (127.26.37.62:36285), 5a028f1f75b94a56adade49eb2e6d92d (127.26.37.60:35025)
I20260809 07:48:55.139539 31780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "950ff7120dbc4fbebd2065da8b3adc57" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5a028f1f75b94a56adade49eb2e6d92d" is_pre_election: true
I20260809 07:48:55.139858 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "950ff7120dbc4fbebd2065da8b3adc57" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cbf5887cb00342e19f5e051f71bb04a2" is_pre_election: true
I20260809 07:48:55.140309 32432 raft_consensus.cc:2393] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 950ff7120dbc4fbebd2065da8b3adc57 in current term 4: Already voted for candidate 5a028f1f75b94a56adade49eb2e6d92d in this term.
I20260809 07:48:55.141057 31632 leader_election.cc:304] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 950ff7120dbc4fbebd2065da8b3adc57; no voters: 5a028f1f75b94a56adade49eb2e6d92d, cbf5887cb00342e19f5e051f71bb04a2
I20260809 07:48:55.141501 32672 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:48:55.143805 32672 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:48:55.209844 32194 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:55.210071 32194 heartbeater.cc:499] Master 127.26.37.62:36285 was elected leader, sending a full tablet report...
I20260809 07:48:55.211012 32419 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e634609e6b2347198ac3ad7b4ac22823" instance_seqno: 1786261733151411) as {username='slave'} at 127.26.37.2:53637; Asking this server to re-register.
I20260809 07:48:55.211709 32194 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:55.212884 32419 ts_manager.cc:194] Registered new tserver with Master: e634609e6b2347198ac3ad7b4ac22823 (127.26.37.2:46637)
W20260809 07:48:55.419409 32671 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:55.419723 32671 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:55.419875 32671 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:48:55.419963 32671 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:55.428102 32671 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:55.428339 32671 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:55.438310 32671 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:40917
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=43205
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:55.440207 32671 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:55.442144 32671 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:55.452173 32684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:55.452211 32682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:55.454310 32687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:55.454988 32671 server_base.cc:1061] running on GCE node
I20260809 07:48:55.455518 32671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:55.456363 32671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:55.457595 32671 hybrid_clock.cc:648] HybridClock initialized: now 1786261735457533 us; error 28 us; skew 500 ppm
I20260809 07:48:55.457864 32671 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:55.459990 32671 webserver.cc:533] Webserver started at http://127.26.37.1:43205/ using document root <none> and password file <none>
I20260809 07:48:55.460651 32671 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:55.460764 32671 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:55.466159 32671 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 07:48:55.468997 32695 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.470337 32671 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:48:55.470460 32671 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:55.471261 32671 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:55.494169 31672 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [term 4 FOLLOWER]: Refusing update from remote peer 5a028f1f75b94a56adade49eb2e6d92d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 07:48:55.495042 32667 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [LEADER]: Connected to new peer: Peer: permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:48:55.497015 32672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a028f1f75b94a56adade49eb2e6d92d. Latest consensus state: current_term: 4 leader_uuid: "5a028f1f75b94a56adade49eb2e6d92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:55.497279 32672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:55.497846 32671 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:55.499027 32671 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:55.499042 32665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "5a028f1f75b94a56adade49eb2e6d92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:55.499303 32665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:55.499466 32671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:55.500279 32432 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [term 4 FOLLOWER]: Refusing update from remote peer 5a028f1f75b94a56adade49eb2e6d92d: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 07:48:55.501016 32671 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:55.501085 32667 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:48:55.502043 32672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "5a028f1f75b94a56adade49eb2e6d92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:55.502264 32672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950ff7120dbc4fbebd2065da8b3adc57 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:55.503551 32671 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:55.503686 32671 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.503826 32671 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:55.503945 32671 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:48:55.505683 32671 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:48:55.505844 32671 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:48:55.505936 32671 fault_injection.cc:59] injecting fault for kudu (pid 32671): FLAGS_fault_before_start (process will exit)
I20260809 07:48:55.505905 32712 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "5a028f1f75b94a56adade49eb2e6d92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:55.506673 32707 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a028f1f75b94a56adade49eb2e6d92d. Latest consensus state: current_term: 4 leader_uuid: "5a028f1f75b94a56adade49eb2e6d92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:55.509229 32712 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:55.509297 32707 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbf5887cb00342e19f5e051f71bb04a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:55.533016 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:40917
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--webserver_port=43205
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:46609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:48:55.878355 32713 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:55.878655 32713 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:55.878839 32713 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:55.887035 32713 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:55.887235 32713 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:55.896956 32713 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:40917
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=43205
--tserver_master_addrs=127.26.37.62:36285,127.26.37.61:42757,127.26.37.60:35025
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:55.898638 32713 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:55.900987 32713 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:55.911427 32719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:55.911619 32718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:55.912665 32721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:55.913137 32713 server_base.cc:1061] running on GCE node
I20260809 07:48:55.913746 32713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:55.914738 32713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:55.916025 32713 hybrid_clock.cc:648] HybridClock initialized: now 1786261735915947 us; error 63 us; skew 500 ppm
I20260809 07:48:55.916340 32713 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:55.918953 32713 webserver.cc:533] Webserver started at http://127.26.37.1:43205/ using document root <none> and password file <none>
I20260809 07:48:55.919804 32713 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:55.919929 32713 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:55.925812 32667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "5a028f1f75b94a56adade49eb2e6d92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf5887cb00342e19f5e051f71bb04a2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36285 } } peers { permanent_uuid: "950ff7120dbc4fbebd2065da8b3adc57" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42757 } } peers { permanent_uuid: "5a028f1f75b94a56adade49eb2e6d92d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35025 } } }
I20260809 07:48:55.926028 32667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a028f1f75b94a56adade49eb2e6d92d [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:55.926451 32713 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 07:48:55.929512 32731 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.930801 32713 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:48:55.930964 32713 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "fe9e153fa1d141e084b1f0d06d4064a2"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-bw88"
I20260809 07:48:55.931754 32713 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:55.951795 32713 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:55.952659 32713 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:55.952998 32713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:55.954053 32713 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:55.955796 32713 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:55.955904 32713 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.956069 32713 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:55.956140 32713 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.991643 32713 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:40917
I20260809 07:48:55.991744 404 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:40917 every 8 connection(s)
I20260809 07:48:55.993207 32713 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:55.994408 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32713
I20260809 07:48:55.994727 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32713
I20260809 07:48:56.023617 400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:56.108139 405 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36285
I20260809 07:48:56.108703 405 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:56.110208 405 heartbeater.cc:507] Master 127.26.37.62:36285 requested a full tablet report, sending...
I20260809 07:48:56.111496 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32713
I20260809 07:48:56.113399 32419 ts_manager.cc:194] Re-registered known tserver with Master: fe9e153fa1d141e084b1f0d06d4064a2 (127.26.37.1:40917)
W20260809 07:48:56.134684 32396 connection.cc:573] server connection from 127.26.37.1:33509 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:48:56.135538 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32034
W20260809 07:48:56.135852 419 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:48:56.012564 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:37407 (local address 127.26.37.61:42757)
0809 07:48:56.021529 (+ 8965us) server_negotiation.cc:207] Beginning negotiation
0809 07:48:56.021537 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:48:56.021556 (+ 19us) server_negotiation.cc:408] Connection header received
0809 07:48:56.022237 (+ 681us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:48:56.022247 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:48:56.022338 (+ 91us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.022480 (+ 142us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:48:56.023863 (+ 1383us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.025722 (+ 1859us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.110161 (+ 84439us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.111124 (+ 963us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.113090 (+ 1966us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0809 07:48:56.113120 (+ 30us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0809 07:48:56.113123 (+ 3us) server_negotiation.cc:893] Client requested to use mechanism: PLAIN
0809 07:48:56.113139 (+ 16us) server_negotiation.cc:923] calling sasl_server_start()
0809 07:48:56.113178 (+ 39us) server_negotiation.cc:1092] Received PLAIN auth, user=slave
0809 07:48:56.113191 (+ 13us) server_negotiation.cc:378] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:56.113253 (+ 62us) server_negotiation.cc:1036] Waiting for connection context
0809 07:48:56.134738 (+ 21485us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:37407: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":8835,"thread_start_us":8215,"threads_started":1}
I20260809 07:48:56.143479 32188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:56.230715 32195 heartbeater.cc:499] Master 127.26.37.60:35025 was elected leader, sending a full tablet report...
I20260809 07:48:56.237156 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32034
I20260809 07:48:56.261905 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32202
I20260809 07:48:56.270571 32359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:56.363693 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32202
I20260809 07:48:56.390036 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32374
I20260809 07:48:56.396359 32458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:56.502437 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32374
I20260809 07:48:56.531824 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31592
I20260809 07:48:56.533219 31695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:56.642061 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31592
I20260809 07:48:56.671290 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31703
I20260809 07:48:56.672549 31810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:48:56.776386 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31703
2026-08-09T07:48:56Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/1 (13561 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/2
2026-08-09T07:48:56Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:56Z Disabled control of system clock
2026-08-09T07:48:56Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:56Z Disabled control of system clock
2026-08-09T07:48:56Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:56Z Disabled control of system clock
2026-08-09T07:48:56Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:56Z Disabled control of system clock
2026-08-09T07:48:56Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:56Z Disabled control of system clock
I20260809 07:48:56.946593 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34559
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471 with env {}
W20260809 07:48:57.289261 444 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:57.289621 444 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:57.289690 444 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:57.297878 444 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:57.297972 444 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:57.298029 444 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:57.298071 444 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:57.308071 444 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:57.309715 444 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:57.311501 444 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:57.319763 451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:57.319808 452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:57.320683 454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:57.321698 444 server_base.cc:1061] running on GCE node
I20260809 07:48:57.322500 444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:57.324045 444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:57.325510 444 hybrid_clock.cc:648] HybridClock initialized: now 1786261737325301 us; error 171 us; skew 500 ppm
I20260809 07:48:57.325862 444 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:57.327913 444 webserver.cc:533] Webserver started at http://127.26.37.62:38879/ using document root <none> and password file <none>
I20260809 07:48:57.328639 444 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:57.328779 444 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:57.329129 444 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:57.331593 444 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "aa24f1d5861849978bc80d09961c6796"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:48:57.332391 444 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "aa24f1d5861849978bc80d09961c6796"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:48:57.337738 444 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 07:48:57.341132 461 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:57.342650 444 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:57.342844 444 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "aa24f1d5861849978bc80d09961c6796"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:48:57.343080 444 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:57.357612 444 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:57.358423 444 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:57.358717 444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:57.379333 444 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34559
I20260809 07:48:57.379359 522 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34559 every 8 connection(s)
I20260809 07:48:57.380846 444 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:57.383088 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 444
I20260809 07:48:57.383307 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:48:57.384311 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:35057
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471 with env {}
I20260809 07:48:57.385432 524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:57.391223 524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:57.408221 524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:57.409529 467 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35057: connect: Connection refused (error 111)
W20260809 07:48:57.411300 524 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35057: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35057: connect: Connection refused (error 111)
I20260809 07:48:57.460697 524 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } attempt: 1
W20260809 07:48:57.463318 524 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35057: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35057: connect: Connection refused (error 111)
I20260809 07:48:57.531610 524 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } attempt: 2
W20260809 07:48:57.533957 524 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35057: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35057: connect: Connection refused (error 111)
I20260809 07:48:57.625262 524 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } attempt: 3
W20260809 07:48:57.627475 524 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35057: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35057: connect: Connection refused (error 111)
W20260809 07:48:57.734988 526 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:57.735348 526 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:57.735539 526 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:57.743525 526 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:57.743614 526 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:57.743705 526 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:57.743795 526 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:57.753718 526 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:35057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:57.755330 526 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:57.757143 526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:57.765527 535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:57.765610 536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:57.766124 538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:57.768312 526 server_base.cc:1061] running on GCE node
I20260809 07:48:57.769107 526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:57.770737 526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:57.770754 524 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } attempt: 4
I20260809 07:48:57.772123 526 hybrid_clock.cc:648] HybridClock initialized: now 1786261737771922 us; error 164 us; skew 500 ppm
I20260809 07:48:57.772571 526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260809 07:48:57.772975 524 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35057: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35057: connect: Connection refused (error 111)
I20260809 07:48:57.774943 526 webserver.cc:533] Webserver started at http://127.26.37.61:42621/ using document root <none> and password file <none>
I20260809 07:48:57.775697 526 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:57.775826 526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:57.776214 526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:57.778666 526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "ee8bafe05bd540e0a9a1d488adabdee8"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:48:57.779500 526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "ee8bafe05bd540e0a9a1d488adabdee8"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:48:57.784808 526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 07:48:57.788242 549 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:57.789633 526 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:57.789813 526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "ee8bafe05bd540e0a9a1d488adabdee8"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:48:57.790040 526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:57.806023 526 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:57.806818 526 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:57.807106 526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:57.830361 526 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:35057
I20260809 07:48:57.830380 611 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:35057 every 8 connection(s)
I20260809 07:48:57.831928 526 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:48:57.836404 613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:57.837329 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 526
I20260809 07:48:57.837566 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:48:57.839020 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:36471
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471 with env {}
I20260809 07:48:57.845688 613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:57.859289 613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:57.868894 613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:57.869979 552 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
W20260809 07:48:57.871654 613 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
I20260809 07:48:57.921074 613 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } attempt: 1
W20260809 07:48:57.923462 613 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
I20260809 07:48:57.991777 613 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } attempt: 2
W20260809 07:48:57.994438 613 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
I20260809 07:48:58.072252 524 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } attempt: 5
I20260809 07:48:58.079684 524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:48:58.081712 524 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
I20260809 07:48:58.085685 613 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } attempt: 3
W20260809 07:48:58.087579 613 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
I20260809 07:48:58.132930 524 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } attempt: 1
W20260809 07:48:58.135689 524 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
W20260809 07:48:58.195581 617 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:58.195976 617 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:58.196113 617 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:48:58.203914 524 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } attempt: 2
W20260809 07:48:58.205651 617 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:58.205765 617 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:58.205855 617 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:58.205919 617 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260809 07:48:58.206154 524 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
I20260809 07:48:58.216066 617 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:36471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:58.217943 617 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:58.219745 617 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:58.227609 625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:58.227739 626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:58.228592 628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:58.229645 617 server_base.cc:1061] running on GCE node
I20260809 07:48:58.230343 617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:58.230823 613 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } attempt: 4
I20260809 07:48:58.232049 617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260809 07:48:58.233029 613 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111)
I20260809 07:48:58.233408 617 hybrid_clock.cc:648] HybridClock initialized: now 1786261738233248 us; error 125 us; skew 500 ppm
I20260809 07:48:58.233793 617 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:58.235775 617 webserver.cc:533] Webserver started at http://127.26.37.60:46693/ using document root <none> and password file <none>
I20260809 07:48:58.236562 617 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:58.236682 617 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:58.237088 617 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:58.239514 617 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "0b75351d11604db78444ee9c04118d04"
format_stamp: "Formatted at 2026-08-09 07:48:58 on dist-test-slave-bw88"
I20260809 07:48:58.240341 617 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "0b75351d11604db78444ee9c04118d04"
format_stamp: "Formatted at 2026-08-09 07:48:58 on dist-test-slave-bw88"
I20260809 07:48:58.245857 617 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 07:48:58.249421 635 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:58.250698 617 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:58.250887 617 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "0b75351d11604db78444ee9c04118d04"
format_stamp: "Formatted at 2026-08-09 07:48:58 on dist-test-slave-bw88"
I20260809 07:48:58.251104 617 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:58.277850 617 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:58.278716 617 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:58.279016 617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:58.299116 617 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:36471
I20260809 07:48:58.299139 706 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:36471 every 8 connection(s)
I20260809 07:48:58.300623 617 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:48:58.302785 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 617
I20260809 07:48:58.303023 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:48:58.305125 709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:58.312454 524 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } attempt: 3
I20260809 07:48:58.313839 709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:58.328122 709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:58.331367 524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap starting.
I20260809 07:48:58.334527 709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:58.335526 524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:58.336910 524 log.cc:826] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:58.339581 524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: No bootstrap required, opened a new log
I20260809 07:48:58.344687 709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Bootstrap starting.
I20260809 07:48:58.345521 524 raft_consensus.cc:359] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.345781 524 raft_consensus.cc:385] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:58.345906 524 raft_consensus.cc:740] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa24f1d5861849978bc80d09961c6796, State: Initialized, Role: FOLLOWER
I20260809 07:48:58.346462 524 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.347405 716 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.347734 716 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:58.348105 709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:58.348624 524 sys_catalog.cc:565] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:58.349269 709 log.cc:826] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:58.353895 709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: No bootstrap required, opened a new log
W20260809 07:48:58.360080 731 catalog_manager.cc:1594] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:58.360193 731 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:58.360896 709 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.361285 709 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 07:48:58.361311 469 tablet.cc:2406] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:58.361436 709 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b75351d11604db78444ee9c04118d04, State: Initialized, Role: FOLLOWER
I20260809 07:48:58.362136 709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.359799 444 master_runner.cc:187] Error getting master registration for 127.26.37.61:35057: OK, instance_id { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" instance_seqno: 1786261737822447 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:48:58.367671 734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.368009 734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:58.368206 444 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:58.368850 709 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:48:58.379351 752 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:58.379514 752 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:58.381304 617 master_runner.cc:187] Error getting master registration for 127.26.37.61:35057: OK, instance_id { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" instance_seqno: 1786261737822447 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:48:58.388897 617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:48:58.498925 734 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:58.499257 734 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.501097 734 leader_election.cc:290] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa24f1d5861849978bc80d09961c6796 (127.26.37.62:34559), ee8bafe05bd540e0a9a1d488adabdee8 (127.26.37.61:35057)
I20260809 07:48:58.501902 493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b75351d11604db78444ee9c04118d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa24f1d5861849978bc80d09961c6796" is_pre_election: true
I20260809 07:48:58.502425 493 raft_consensus.cc:2468] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b75351d11604db78444ee9c04118d04 in term 0.
I20260809 07:48:58.503099 639 leader_election.cc:304] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b75351d11604db78444ee9c04118d04, aa24f1d5861849978bc80d09961c6796; no voters:
I20260809 07:48:58.503532 734 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:58.503685 734 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:58.503784 734 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:58.507395 734 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.508625 493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b75351d11604db78444ee9c04118d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa24f1d5861849978bc80d09961c6796"
I20260809 07:48:58.508711 734 leader_election.cc:290] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [CANDIDATE]: Term 1 election: Requested vote from peers aa24f1d5861849978bc80d09961c6796 (127.26.37.62:34559), ee8bafe05bd540e0a9a1d488adabdee8 (127.26.37.61:35057)
I20260809 07:48:58.508831 493 raft_consensus.cc:3060] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:58.512445 493 raft_consensus.cc:2468] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b75351d11604db78444ee9c04118d04 in term 1.
I20260809 07:48:58.512980 639 leader_election.cc:304] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b75351d11604db78444ee9c04118d04, aa24f1d5861849978bc80d09961c6796; no voters:
I20260809 07:48:58.513448 734 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:58.518137 734 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 1 LEADER]: Becoming Leader. State: Replica: 0b75351d11604db78444ee9c04118d04, State: Running, Role: LEADER
I20260809 07:48:58.519301 734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.526069 756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b75351d11604db78444ee9c04118d04. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.526389 756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:58.528680 759 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
W20260809 07:48:58.531049 642 tablet.cc:2406] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:58.532274 613 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } attempt: 5
I20260809 07:48:58.532415 759 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:48:58.547199 613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: Bootstrap starting.
I20260809 07:48:58.547760 493 raft_consensus.cc:1275] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Refusing update from remote peer 0b75351d11604db78444ee9c04118d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:58.548957 734 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:58.559072 613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:58.561396 613 log.cc:826] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:58.575090 613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: No bootstrap required, opened a new log
I20260809 07:48:58.577463 716 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b75351d11604db78444ee9c04118d04. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.577731 716 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:58.578964 760 mvcc.cc:204] Tried to move back new op lower bound from 7316528081077665792 to 7316528080986198016. Current Snapshot: MvccSnapshot[applied={T|T < 7316528081077665792}]
I20260809 07:48:58.586455 716 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.586686 716 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:58.593590 613 raft_consensus.cc:359] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.594230 613 raft_consensus.cc:385] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:58.594512 613 raft_consensus.cc:740] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8bafe05bd540e0a9a1d488adabdee8, State: Initialized, Role: FOLLOWER
I20260809 07:48:58.594501 734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.594784 734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:58.596349 613 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:48:58.596735 759 catalog_manager.cc:1383] Generated new cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:48:58.596848 759 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:48:58.598037 768 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.598489 768 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:58.602869 580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b75351d11604db78444ee9c04118d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" is_pre_election: true
I20260809 07:48:58.602713 579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b75351d11604db78444ee9c04118d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee8bafe05bd540e0a9a1d488adabdee8"
I20260809 07:48:58.603940 578 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:58.605402 613 sys_catalog.cc:565] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:58.608095 578 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 1 FOLLOWER]: Refusing update from remote peer 0b75351d11604db78444ee9c04118d04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:58.609166 756 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:58.667565 768 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b75351d11604db78444ee9c04118d04. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.667933 768 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:58.668403 774 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.668697 774 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:58.669811 756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:48:58.670059 756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:58.673957 759 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:48:58.675770 759 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:48:58.707180 759 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Generated new TSK 0
I20260809 07:48:58.708356 759 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:48:58.730239 788 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:48:58.730353 788 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: loading cluster ID for follower catalog manager: success
I20260809 07:48:58.739289 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:58.749096 788 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: acquiring CA information for follower catalog manager: success
I20260809 07:48:58.752137 788 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:48:59.215137 791 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:59.215487 791 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:59.215610 791 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:59.224315 791 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:59.224598 791 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:48:59.234781 791 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:59.236397 791 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:59.238494 791 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:59.248567 799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:59.248838 798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:59.249588 801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:59.249863 791 server_base.cc:1061] running on GCE node
I20260809 07:48:59.250514 791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:59.251645 791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:59.252894 791 hybrid_clock.cc:648] HybridClock initialized: now 1786261739252749 us; error 111 us; skew 500 ppm
I20260809 07:48:59.253250 791 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:59.255403 791 webserver.cc:533] Webserver started at http://127.26.37.1:38309/ using document root <none> and password file <none>
I20260809 07:48:59.256219 791 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:59.256367 791 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:59.256754 791 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:59.263161 791 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:48:59.264038 791 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:48:59.269778 791 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:48:59.273640 808 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.275048 791 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:59.275194 791 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:48:59.275370 791 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:59.301433 791 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:59.302394 791 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:59.302686 791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:59.303642 791 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:59.305291 791 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:59.305433 791 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 07:48:59.305661 791 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:59.305732 791 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.347319 791 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39789
I20260809 07:48:59.347422 953 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39789 every 8 connection(s)
I20260809 07:48:59.348912 791 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:59.357496 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 791
I20260809 07:48:59.357762 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:59.362412 731 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:48:59.362522 731 catalog_manager.cc:1588] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: loading cluster ID for follower catalog manager: success
I20260809 07:48:59.365515 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:59.365288 731 catalog_manager.cc:1610] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: acquiring CA information for follower catalog manager: success
I20260809 07:48:59.367895 731 catalog_manager.cc:1638] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:59.380883 957 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:48:59.381273 957 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:59.382378 957 heartbeater.cc:507] Master 127.26.37.61:35057 requested a full tablet report, sending...
I20260809 07:48:59.386458 569 ts_manager.cc:194] Registered new tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:39789)
I20260809 07:48:59.389113 955 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36471
I20260809 07:48:59.389263 955 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:59.389894 955 heartbeater.cc:507] Master 127.26.37.60:36471 requested a full tablet report, sending...
I20260809 07:48:59.391726 656 ts_manager.cc:194] Registered new tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:39789)
I20260809 07:48:59.393818 656 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:52607
I20260809 07:48:59.400117 956 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:48:59.400262 956 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:59.400589 956 heartbeater.cc:507] Master 127.26.37.62:34559 requested a full tablet report, sending...
I20260809 07:48:59.402544 480 ts_manager.cc:194] Registered new tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:39789)
W20260809 07:48:59.729405 968 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:59.729759 968 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:59.729846 968 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:59.738018 968 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:59.738166 968 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:48:59.747757 968 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:48:59.749429 968 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:59.751266 968 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:48:59.761264 976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:59.761198 975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:59.766523 978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:59.767161 968 server_base.cc:1061] running on GCE node
I20260809 07:48:59.767761 968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:59.768868 968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:59.770108 968 hybrid_clock.cc:648] HybridClock initialized: now 1786261739769939 us; error 130 us; skew 500 ppm
I20260809 07:48:59.770494 968 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:48:59.773144 968 webserver.cc:533] Webserver started at http://127.26.37.2:43641/ using document root <none> and password file <none>
I20260809 07:48:59.773977 968 fs_manager.cc:362] Metadata directory not provided
I20260809 07:48:59.774127 968 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:59.774510 968 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:59.776966 968 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "af888a1cd1f8488a8f0d165a1fb0a12b"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:48:59.777787 968 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "af888a1cd1f8488a8f0d165a1fb0a12b"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:48:59.783488 968 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:48:59.786883 984 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.788173 968 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:48:59.788367 968 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "af888a1cd1f8488a8f0d165a1fb0a12b"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:48:59.788591 968 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:59.829942 968 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:48:59.830843 968 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:59.831151 968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:59.832208 968 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:48:59.834263 968 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:59.834404 968 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.834545 968 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:48:59.834626 968 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.872167 968 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:39277
I20260809 07:48:59.872252 1139 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:39277 every 8 connection(s)
I20260809 07:48:59.873685 968 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:59.882702 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 968
I20260809 07:48:59.882951 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:59.888108 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:48:59.890698 1140 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36471
I20260809 07:48:59.890807 1141 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:48:59.891140 1140 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:59.891146 1141 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:59.892062 1140 heartbeater.cc:507] Master 127.26.37.60:36471 requested a full tablet report, sending...
I20260809 07:48:59.892112 1141 heartbeater.cc:507] Master 127.26.37.62:34559 requested a full tablet report, sending...
I20260809 07:48:59.893806 656 ts_manager.cc:194] Registered new tserver with Master: af888a1cd1f8488a8f0d165a1fb0a12b (127.26.37.2:39277)
I20260809 07:48:59.893982 480 ts_manager.cc:194] Registered new tserver with Master: af888a1cd1f8488a8f0d165a1fb0a12b (127.26.37.2:39277)
I20260809 07:48:59.894824 656 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:44319
I20260809 07:48:59.899081 1143 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:48:59.899233 1143 heartbeater.cc:461] Registering TS with master...
I20260809 07:48:59.899673 1143 heartbeater.cc:507] Master 127.26.37.61:35057 requested a full tablet report, sending...
I20260809 07:48:59.900696 569 ts_manager.cc:194] Registered new tserver with Master: af888a1cd1f8488a8f0d165a1fb0a12b (127.26.37.2:39277)
W20260809 07:49:00.263607 1151 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:00.263923 1151 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:00.263996 1151 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:00.272212 1151 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:00.272406 1151 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:49:00.282474 1151 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:00.284103 1151 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:00.286039 1151 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:00.295634 1159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.295605 1157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.297432 1161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:00.298302 1151 server_base.cc:1061] running on GCE node
I20260809 07:49:00.299037 1151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:00.300414 1151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:00.301731 1151 hybrid_clock.cc:648] HybridClock initialized: now 1786261740301548 us; error 145 us; skew 500 ppm
I20260809 07:49:00.302084 1151 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:00.304563 1151 webserver.cc:533] Webserver started at http://127.26.37.3:44195/ using document root <none> and password file <none>
I20260809 07:49:00.305531 1151 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:00.305648 1151 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:00.306038 1151 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:00.309211 1151 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "b5b89a7e8b94446abeed2bc4318ffa32"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-bw88"
I20260809 07:49:00.310238 1151 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "b5b89a7e8b94446abeed2bc4318ffa32"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-bw88"
I20260809 07:49:00.315769 1151 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:49:00.319208 1170 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.320647 1151 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:49:00.320837 1151 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "b5b89a7e8b94446abeed2bc4318ffa32"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-bw88"
I20260809 07:49:00.321030 1151 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:00.353575 1151 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:00.354651 1151 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:00.354947 1151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:00.356220 1151 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:00.358270 1151 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:00.358361 1151 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.358465 1151 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:00.358522 1151 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.396091 1151 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:43579
I20260809 07:49:00.396158 1312 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:43579 every 8 connection(s)
I20260809 07:49:00.396833 955 heartbeater.cc:499] Master 127.26.37.60:36471 was elected leader, sending a full tablet report...
I20260809 07:49:00.398363 1151 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:00.405406 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1151
I20260809 07:49:00.405649 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:49:00.419456 1314 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:49:00.419812 1314 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:00.420734 1315 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:49:00.420825 1314 heartbeater.cc:507] Master 127.26.37.62:34559 requested a full tablet report, sending...
I20260809 07:49:00.420928 1315 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:00.421635 1315 heartbeater.cc:507] Master 127.26.37.61:35057 requested a full tablet report, sending...
I20260809 07:49:00.423118 480 ts_manager.cc:194] Registered new tserver with Master: b5b89a7e8b94446abeed2bc4318ffa32 (127.26.37.3:43579)
I20260809 07:49:00.423138 569 ts_manager.cc:194] Registered new tserver with Master: b5b89a7e8b94446abeed2bc4318ffa32 (127.26.37.3:43579)
I20260809 07:49:00.424067 1313 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36471
I20260809 07:49:00.424201 1313 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:00.424592 1313 heartbeater.cc:507] Master 127.26.37.60:36471 requested a full tablet report, sending...
I20260809 07:49:00.425544 656 ts_manager.cc:194] Registered new tserver with Master: b5b89a7e8b94446abeed2bc4318ffa32 (127.26.37.3:43579)
I20260809 07:49:00.426632 656 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:43633
I20260809 07:49:00.431025 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:49:00.437929 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:49:00.438115 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 444
I20260809 07:49:00.439002 516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:00.548254 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 444
I20260809 07:49:00.578958 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34559
--webserver_interface=127.26.37.62
--webserver_port=38879
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471 with env {}
W20260809 07:49:00.618260 639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:34559: connect: Connection refused (error 111)
W20260809 07:49:00.619840 639 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 -> Peer aa24f1d5861849978bc80d09961c6796 (127.26.37.62:34559): Couldn't send request to peer aa24f1d5861849978bc80d09961c6796. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:34559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:49:00.896764 1141 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:34559 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:34559: connect: Connection refused (error 111)
I20260809 07:49:00.897537 1140 heartbeater.cc:499] Master 127.26.37.60:36471 was elected leader, sending a full tablet report...
W20260809 07:49:00.930783 1329 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:00.931119 1329 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:00.931247 1329 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:00.939375 1329 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:00.939502 1329 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:00.939621 1329 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:00.939704 1329 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:00.949292 1329 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=38879
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:00.950836 1329 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:00.952684 1329 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:00.960954 1338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.961817 1339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.961848 1342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:00.962915 1329 server_base.cc:1061] running on GCE node
I20260809 07:49:00.963670 1329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:00.965142 1329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:00.966342 1329 hybrid_clock.cc:648] HybridClock initialized: now 1786261740966206 us; error 107 us; skew 500 ppm
I20260809 07:49:00.966694 1329 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:00.968791 1329 webserver.cc:533] Webserver started at http://127.26.37.62:38879/ using document root <none> and password file <none>
I20260809 07:49:00.969564 1329 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:00.969722 1329 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:00.975145 1329 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:49:00.977890 1349 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.979264 1329 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:00.979429 1329 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "aa24f1d5861849978bc80d09961c6796"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:49:00.980121 1329 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:01.010283 1329 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:01.011456 1329 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:01.011843 1329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:01.032774 1329 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34559
I20260809 07:49:01.032811 1416 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34559 every 8 connection(s)
I20260809 07:49:01.034519 1329 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:01.040251 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1329
I20260809 07:49:01.040602 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 791
I20260809 07:49:01.042008 1419 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:01.046082 1419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap starting.
I20260809 07:49:01.050709 949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:01.065518 1419 log.cc:826] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:01.074497 1419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:01.075065 1419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap complete.
I20260809 07:49:01.081921 1419 raft_consensus.cc:359] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:01.082669 1419 raft_consensus.cc:740] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa24f1d5861849978bc80d09961c6796, State: Initialized, Role: FOLLOWER
I20260809 07:49:01.083508 1419 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:01.084414 1429 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:01.084712 1429 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:01.085963 1419 sys_catalog.cc:565] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:01.113306 1447 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:49:01.113447 1447 catalog_manager.cc:1588] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: loading cluster ID for follower catalog manager: success
I20260809 07:49:01.125906 756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:01.126560 756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:01.130924 1447 catalog_manager.cc:1610] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: acquiring CA information for follower catalog manager: success
I20260809 07:49:01.133715 1447 catalog_manager.cc:1638] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:01.145977 1429 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b75351d11604db78444ee9c04118d04. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:01.146286 1429 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:01.177300 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 791
I20260809 07:49:01.203569 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39789
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--webserver_port=38309
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:49:01.428854 1313 heartbeater.cc:499] Master 127.26.37.60:36471 was elected leader, sending a full tablet report...
I20260809 07:49:01.437664 1367 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b5b89a7e8b94446abeed2bc4318ffa32" instance_seqno: 1786261740385294) as {username='slave'} at 127.26.37.3:47987; Asking this server to re-register.
I20260809 07:49:01.438591 1314 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:01.440351 1367 ts_manager.cc:194] Registered new tserver with Master: b5b89a7e8b94446abeed2bc4318ffa32 (127.26.37.3:43579)
W20260809 07:49:01.563226 1460 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:01.563575 1460 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:01.563723 1460 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:01.572491 1460 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:01.572734 1460 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:01.579013 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b75351d11604db78444ee9c04118d04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:01.579361 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:01.583559 1460 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39789
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38309
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:01.585325 1460 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:01.587348 1460 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:01.597324 1468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:01.597294 1467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:01.597738 1470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:01.598881 1460 server_base.cc:1061] running on GCE node
I20260809 07:49:01.599457 1460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:01.600582 1460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:01.601899 1460 hybrid_clock.cc:648] HybridClock initialized: now 1786261741601752 us; error 112 us; skew 500 ppm
I20260809 07:49:01.602228 1460 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:01.604434 1460 webserver.cc:533] Webserver started at http://127.26.37.1:38309/ using document root <none> and password file <none>
I20260809 07:49:01.605234 1460 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:01.605571 1460 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:01.610991 1460 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:49:01.613651 1479 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.614916 1460 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:49:01.615105 1460 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:49:01.615831 1460 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:01.631295 1460 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:01.632120 1460 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:01.632422 1460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:01.633462 1460 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:01.635138 1460 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:01.635242 1460 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.635393 1460 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:01.635468 1460 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.669787 1460 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39789
I20260809 07:49:01.669826 1631 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39789 every 8 connection(s)
I20260809 07:49:01.671303 1460 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:01.675896 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1460
I20260809 07:49:01.676258 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1460
I20260809 07:49:01.711117 1633 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:49:01.711520 1633 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:01.711624 1634 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:49:01.711805 1634 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:01.712991 1634 heartbeater.cc:507] Master 127.26.37.61:35057 requested a full tablet report, sending...
I20260809 07:49:01.712993 1633 heartbeater.cc:507] Master 127.26.37.62:34559 requested a full tablet report, sending...
I20260809 07:49:01.714339 1622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:01.714993 1367 ts_manager.cc:194] Registered new tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:39789)
I20260809 07:49:01.802922 569 ts_manager.cc:194] Re-registered known tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:39789)
I20260809 07:49:01.804615 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1460
I20260809 07:49:01.828307 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39789
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--webserver_port=38309
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260809 07:49:01.904908 1141 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:49:01.906078 1367 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "af888a1cd1f8488a8f0d165a1fb0a12b" instance_seqno: 1786261739863804) as {username='slave'} at 127.26.37.2:53533; Asking this server to re-register.
I20260809 07:49:01.906641 1141 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:01.907788 1367 ts_manager.cc:194] Registered new tserver with Master: af888a1cd1f8488a8f0d165a1fb0a12b (127.26.37.2:39277)
W20260809 07:49:02.182396 1645 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:02.182704 1645 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:02.182847 1645 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:49:02.182941 1645 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:02.191371 1645 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:02.191603 1645 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:02.201328 1645 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39789
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38309
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:02.202947 1645 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:02.204702 1645 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:02.215133 1654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:02.215263 1655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:02.216274 1659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:02.216722 1645 server_base.cc:1061] running on GCE node
I20260809 07:49:02.217250 1645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:02.218470 1645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:02.219691 1645 hybrid_clock.cc:648] HybridClock initialized: now 1786261742219526 us; error 121 us; skew 500 ppm
I20260809 07:49:02.219997 1645 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:02.222241 1645 webserver.cc:533] Webserver started at http://127.26.37.1:38309/ using document root <none> and password file <none>
I20260809 07:49:02.222982 1645 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:02.223129 1645 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:02.228555 1645 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:49:02.231353 1665 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:02.232818 1645 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:49:02.233016 1645 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:49:02.233786 1645 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:02.269724 1645 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:02.270658 1645 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:02.270953 1645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:02.271991 1645 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:02.274024 1645 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:02.274150 1645 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:02.274289 1645 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:02.274389 1645 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:49:02.274765 1645 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:49:02.274873 1645 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:49:02.274957 1645 fault_injection.cc:59] injecting fault for kudu (pid 1645): FLAGS_fault_before_start (process will exit)
I20260809 07:49:02.303893 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39789
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--webserver_port=38309
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:49:02.660909 1675 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:02.661273 1675 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:02.661446 1675 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:02.670086 1675 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:02.670367 1675 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:02.680547 1675 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:38431,127.26.37.21:35667,127.26.37.22:33799,127.26.37.23:35243,127.26.37.24:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39789
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38309
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:02.682365 1675 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:02.684459 1675 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:02.696084 1683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:02.696144 1682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:02.696406 1687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:02.698951 1675 server_base.cc:1061] running on GCE node
I20260809 07:49:02.699645 1675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:02.700863 1675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:02.702128 1675 hybrid_clock.cc:648] HybridClock initialized: now 1786261742701930 us; error 155 us; skew 500 ppm
I20260809 07:49:02.702471 1675 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:02.704797 1675 webserver.cc:533] Webserver started at http://127.26.37.1:38309/ using document root <none> and password file <none>
I20260809 07:49:02.705726 1675 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:02.705880 1675 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:02.711586 1675 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 07:49:02.714584 1693 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:02.716085 1675 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:49:02.716305 1675 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:49:02.717067 1675 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:02.739845 1675 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:02.740831 1675 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:02.741189 1675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:02.742307 1675 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:02.744153 1675 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:02.744267 1675 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:02.744429 1675 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:02.744509 1675 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:02.781821 1675 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39789
I20260809 07:49:02.781879 1848 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39789 every 8 connection(s)
I20260809 07:49:02.783439 1675 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:02.786803 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1675
I20260809 07:49:02.787154 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1675
I20260809 07:49:02.811411 1850 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:49:02.811450 1851 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:49:02.811869 1851 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:02.811872 1850 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:02.813126 1851 heartbeater.cc:507] Master 127.26.37.61:35057 requested a full tablet report, sending...
I20260809 07:49:02.813198 1849 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36471
I20260809 07:49:02.813318 1849 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:02.813323 1850 heartbeater.cc:507] Master 127.26.37.62:34559 requested a full tablet report, sending...
I20260809 07:49:02.813874 1841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:02.813966 1849 heartbeater.cc:507] Master 127.26.37.60:36471 requested a full tablet report, sending...
I20260809 07:49:02.815214 569 ts_manager.cc:194] Re-registered known tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:39789)
I20260809 07:49:02.815392 656 ts_manager.cc:194] Re-registered known tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:39789)
I20260809 07:49:02.816483 1367 ts_manager.cc:194] Re-registered known tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:39789)
I20260809 07:49:02.816677 656 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:56507
I20260809 07:49:02.907028 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1675
I20260809 07:49:02.933115 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 968
I20260809 07:49:02.941973 1134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:03.028504 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 968
I20260809 07:49:03.054941 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1151
I20260809 07:49:03.064263 1306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:03.154377 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1151
I20260809 07:49:03.180817 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1329
I20260809 07:49:03.187889 1409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:03.292212 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1329
I20260809 07:49:03.322294 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 526
I20260809 07:49:03.323441 607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:03.390708 617 master_runner.cc:187] Error getting master registration for 127.26.37.62:34559: Network error: Client connection negotiation failed: client connection to 127.26.37.62:34559: connect: Connection refused (error 111),
I20260809 07:49:03.390929 617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:49:03.428629 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 526
I20260809 07:49:03.458461 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 617
I20260809 07:49:03.459748 701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:03.566861 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 617
2026-08-09T07:49:03Z chronyd exiting
2026-08-09T07:49:03Z chronyd exiting
2026-08-09T07:49:03Z chronyd exiting
2026-08-09T07:49:03Z chronyd exiting
2026-08-09T07:49:03Z chronyd exiting
2026-08-09T07:49:03Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:03Z Disabled control of system clock
2026-08-09T07:49:03Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:03Z Disabled control of system clock
2026-08-09T07:49:03Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:03Z Disabled control of system clock
2026-08-09T07:49:03Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:03Z Disabled control of system clock
2026-08-09T07:49:03Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:03Z Disabled control of system clock
I20260809 07:49:03.766778 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34559
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471 with env {}
W20260809 07:49:04.104436 1885 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:04.104791 1885 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:04.104948 1885 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:04.113106 1885 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:04.113178 1885 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:04.113278 1885 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:04.113379 1885 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:04.122720 1885 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:04.124176 1885 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:04.125941 1885 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:04.135531 1891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:04.135839 1893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:04.136550 1895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:04.136981 1885 server_base.cc:1061] running on GCE node
I20260809 07:49:04.137773 1885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:04.139231 1885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:04.140462 1885 hybrid_clock.cc:648] HybridClock initialized: now 1786261744140313 us; error 122 us; skew 500 ppm
I20260809 07:49:04.140838 1885 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:04.143174 1885 webserver.cc:533] Webserver started at http://127.26.37.62:32775/ using document root <none> and password file <none>
I20260809 07:49:04.143919 1885 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:04.144073 1885 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:04.149399 1885 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:04.152477 1903 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:04.153844 1885 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 07:49:04.154068 1885 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "aa24f1d5861849978bc80d09961c6796"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:49:04.154850 1885 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:04.176106 1885 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:04.176905 1885 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:04.177194 1885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:04.197330 1885 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34559
I20260809 07:49:04.197422 1974 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34559 every 8 connection(s)
I20260809 07:49:04.198961 1885 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:04.205984 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1885
I20260809 07:49:04.206152 1975 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:04.206189 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:49:04.207499 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:35057
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471 with env {}
I20260809 07:49:04.210938 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap starting.
I20260809 07:49:04.228138 1975 log.cc:826] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:04.236394 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:04.236950 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap complete.
I20260809 07:49:04.243705 1975 raft_consensus.cc:359] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:04.244381 1975 raft_consensus.cc:740] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa24f1d5861849978bc80d09961c6796, State: Initialized, Role: FOLLOWER
I20260809 07:49:04.245168 1975 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:04.246112 1980 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:04.246439 1980 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:04.247149 1975 sys_catalog.cc:565] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:04.257576 1999 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:49:04.257745 1999 catalog_manager.cc:1588] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: loading cluster ID for follower catalog manager: success
I20260809 07:49:04.262853 1999 catalog_manager.cc:1610] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: acquiring CA information for follower catalog manager: success
I20260809 07:49:04.265694 1999 catalog_manager.cc:1638] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:04.278045 1885 master_runner.cc:187] Error getting master registration for 127.26.37.61:35057: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35057: connect: Connection refused (error 111),
I20260809 07:49:04.281858 1885 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:49:04.578430 1977 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:04.578804 1977 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:04.578928 1977 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:04.587908 1977 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:04.587994 1977 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:04.588083 1977 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:04.588155 1977 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:04.598089 1977 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:35057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:04.599881 1977 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:04.601830 1977 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:04.610005 2012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:04.609853 2014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:04.612149 2016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:04.612407 1977 server_base.cc:1061] running on GCE node
I20260809 07:49:04.613154 1977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:04.614801 1977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:04.616051 1977 hybrid_clock.cc:648] HybridClock initialized: now 1786261744615854 us; error 155 us; skew 500 ppm
I20260809 07:49:04.616412 1977 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:04.618678 1977 webserver.cc:533] Webserver started at http://127.26.37.61:34997/ using document root <none> and password file <none>
I20260809 07:49:04.619449 1977 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:04.619614 1977 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:04.625490 1977 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:49:04.628544 2024 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:04.630067 1977 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:04.630265 1977 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "ee8bafe05bd540e0a9a1d488adabdee8"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:49:04.631011 1977 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:04.653306 1977 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:04.654325 1977 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:04.654650 1977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:04.675765 1977 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:35057
I20260809 07:49:04.675791 2086 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:35057 every 8 connection(s)
I20260809 07:49:04.677418 1977 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:49:04.680807 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1977
I20260809 07:49:04.681032 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:49:04.682319 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:36471
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471 with env {}
I20260809 07:49:04.686189 2087 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:04.691022 2087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: Bootstrap starting.
I20260809 07:49:04.707757 2087 log.cc:826] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:04.715829 2087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:04.716336 2087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: Bootstrap complete.
I20260809 07:49:04.722939 2087 raft_consensus.cc:359] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:04.723552 2087 raft_consensus.cc:740] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8bafe05bd540e0a9a1d488adabdee8, State: Initialized, Role: FOLLOWER
I20260809 07:49:04.724215 2087 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:04.725023 2093 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:04.725284 2093 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:04.725971 2087 sys_catalog.cc:565] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:04.735723 2107 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:49:04.735867 2107 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: loading cluster ID for follower catalog manager: success
I20260809 07:49:04.739995 2107 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: acquiring CA information for follower catalog manager: success
I20260809 07:49:04.742398 2107 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:04.769949 1977 master_runner.cc:187] Error getting master registration for 127.26.37.60:36471: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36471: connect: Connection refused (error 111),
I20260809 07:49:04.775868 1977 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:49:05.038458 2090 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:05.038820 2090 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:05.038957 2090 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:05.046960 2090 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:05.047067 2090 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:05.047189 2090 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:05.047271 2090 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:05.057199 2090 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:36471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:05.058779 2090 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:05.060748 2090 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:05.069240 2119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:05.070155 2120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:05.070927 2124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:05.071637 2090 server_base.cc:1061] running on GCE node
I20260809 07:49:05.072429 2090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:05.074137 2090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:05.075397 2090 hybrid_clock.cc:648] HybridClock initialized: now 1786261745075195 us; error 173 us; skew 500 ppm
I20260809 07:49:05.075752 2090 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:05.077984 2090 webserver.cc:533] Webserver started at http://127.26.37.60:33705/ using document root <none> and password file <none>
I20260809 07:49:05.078716 2090 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:05.078855 2090 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:05.084317 2090 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:49:05.090991 2133 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:05.092760 2090 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.002s sys 0.003s
I20260809 07:49:05.092957 2090 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "0b75351d11604db78444ee9c04118d04"
format_stamp: "Formatted at 2026-08-09 07:48:58 on dist-test-slave-bw88"
I20260809 07:49:05.093798 2090 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:05.112774 2090 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:05.113719 2090 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:05.114020 2090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:05.135171 2090 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:36471
I20260809 07:49:05.135296 2204 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:36471 every 8 connection(s)
I20260809 07:49:05.137135 2090 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:49:05.144261 2205 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:05.145741 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2090
I20260809 07:49:05.145933 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:49:05.148005 2205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Bootstrap starting.
I20260809 07:49:05.168005 2205 log.cc:826] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:05.178282 2205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:05.178895 2205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: Bootstrap complete.
I20260809 07:49:05.185501 2205 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:05.186244 2205 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b75351d11604db78444ee9c04118d04, State: Initialized, Role: FOLLOWER
I20260809 07:49:05.186832 2205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:05.187644 2213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:05.187937 2213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:05.188714 2205 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:05.201948 2227 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:49:05.202160 2227 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: loading cluster ID for follower catalog manager: success
I20260809 07:49:05.206518 2227 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: acquiring CA information for follower catalog manager: success
I20260809 07:49:05.211123 2227 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:05.228297 2090 master_runner.cc:264] No leader master found from master 0b75351d11604db78444ee9c04118d04
I20260809 07:49:05.228509 2090 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:49:05.705518 2233 raft_consensus.cc:493] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:49:05.705788 2233 raft_consensus.cc:515] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:05.707618 2233 leader_election.cc:290] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee8bafe05bd540e0a9a1d488adabdee8 (127.26.37.61:35057), 0b75351d11604db78444ee9c04118d04 (127.26.37.60:36471)
I20260809 07:49:05.715104 2174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa24f1d5861849978bc80d09961c6796" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0b75351d11604db78444ee9c04118d04" is_pre_election: true
I20260809 07:49:05.715425 2054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa24f1d5861849978bc80d09961c6796" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" is_pre_election: true
I20260809 07:49:05.715770 2174 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa24f1d5861849978bc80d09961c6796 in term 1.
I20260809 07:49:05.715884 2054 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa24f1d5861849978bc80d09961c6796 in term 1.
I20260809 07:49:05.716465 1906 leader_election.cc:304] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b75351d11604db78444ee9c04118d04, aa24f1d5861849978bc80d09961c6796, ee8bafe05bd540e0a9a1d488adabdee8; no voters:
I20260809 07:49:05.716849 2233 raft_consensus.cc:2804] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:49:05.717005 2233 raft_consensus.cc:493] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:49:05.717131 2233 raft_consensus.cc:3060] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:49:05.720872 2233 raft_consensus.cc:515] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:05.721557 2233 leader_election.cc:290] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [CANDIDATE]: Term 2 election: Requested vote from peers ee8bafe05bd540e0a9a1d488adabdee8 (127.26.37.61:35057), 0b75351d11604db78444ee9c04118d04 (127.26.37.60:36471)
I20260809 07:49:05.721989 2054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa24f1d5861849978bc80d09961c6796" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee8bafe05bd540e0a9a1d488adabdee8"
I20260809 07:49:05.722023 2174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa24f1d5861849978bc80d09961c6796" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0b75351d11604db78444ee9c04118d04"
I20260809 07:49:05.722163 2054 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:49:05.722225 2174 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:49:05.725744 2054 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa24f1d5861849978bc80d09961c6796 in term 2.
I20260809 07:49:05.725991 2174 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa24f1d5861849978bc80d09961c6796 in term 2.
I20260809 07:49:05.726204 1907 leader_election.cc:304] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa24f1d5861849978bc80d09961c6796, ee8bafe05bd540e0a9a1d488adabdee8; no voters:
I20260809 07:49:05.726528 2233 raft_consensus.cc:2804] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:49:05.726943 2233 raft_consensus.cc:697] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 2 LEADER]: Becoming Leader. State: Replica: aa24f1d5861849978bc80d09961c6796, State: Running, Role: LEADER
I20260809 07:49:05.727438 2233 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:05.730317 2238 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa24f1d5861849978bc80d09961c6796. Latest consensus state: current_term: 2 leader_uuid: "aa24f1d5861849978bc80d09961c6796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:05.730551 2238 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:05.731328 2242 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:49:05.733407 2242 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:49:05.734187 2242 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:49:05.734289 2242 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:49:05.735402 2242 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:49:05.736224 2242 catalog_manager.cc:6103] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Loaded TSK: 0
I20260809 07:49:05.737133 2242 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:49:05.749838 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:49:06.095634 2244 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:06.095937 2244 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:06.096069 2244 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:06.104962 2244 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:06.105166 2244 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:06.115200 2244 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:06.116892 2244 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:06.119087 2244 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:06.128952 2251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:06.129545 2250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:06.129873 2253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:06.130923 2244 server_base.cc:1061] running on GCE node
I20260809 07:49:06.131604 2244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:06.132759 2244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:06.134166 2244 hybrid_clock.cc:648] HybridClock initialized: now 1786261746133931 us; error 202 us; skew 500 ppm
I20260809 07:49:06.134649 2244 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:06.137501 2244 webserver.cc:533] Webserver started at http://127.26.37.1:34659/ using document root <none> and password file <none>
I20260809 07:49:06.138505 2244 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:06.138646 2244 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:06.145601 2244 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.004s
I20260809 07:49:06.148849 2261 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:06.150722 2244 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:49:06.150895 2244 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:49:06.151790 2244 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:06.167515 2054 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 2 FOLLOWER]: Refusing update from remote peer aa24f1d5861849978bc80d09961c6796: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:49:06.168601 2238 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:49:06.172585 2270 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa24f1d5861849978bc80d09961c6796. Latest consensus state: current_term: 2 leader_uuid: "aa24f1d5861849978bc80d09961c6796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:06.172916 2270 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:06.174676 2238 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "aa24f1d5861849978bc80d09961c6796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:06.174904 2238 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:06.175122 2270 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "aa24f1d5861849978bc80d09961c6796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:06.175304 2270 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:06.175832 2174 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 2 FOLLOWER]: Refusing update from remote peer aa24f1d5861849978bc80d09961c6796: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:49:06.176708 2238 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:49:06.179524 2244 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:06.180403 2244 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:06.180763 2244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:06.181457 2277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "aa24f1d5861849978bc80d09961c6796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:06.181969 2277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:06.182358 2274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa24f1d5861849978bc80d09961c6796. Latest consensus state: current_term: 2 leader_uuid: "aa24f1d5861849978bc80d09961c6796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:06.182698 2274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:06.182817 2244 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:06.184582 2244 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:06.184657 2244 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:06.184796 2244 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:06.184854 2244 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:06.220444 2244 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34871
I20260809 07:49:06.220538 2422 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34871 every 8 connection(s)
I20260809 07:49:06.222059 2244 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:06.222895 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2244
I20260809 07:49:06.223129 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:49:06.227301 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:49:06.258391 2424 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:49:06.258406 2423 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36471
I20260809 07:49:06.258301 2425 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:49:06.258941 2423 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:06.258941 2424 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:06.258955 2425 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:06.260197 2423 heartbeater.cc:507] Master 127.26.37.60:36471 requested a full tablet report, sending...
I20260809 07:49:06.260227 2424 heartbeater.cc:507] Master 127.26.37.62:34559 requested a full tablet report, sending...
I20260809 07:49:06.260382 2425 heartbeater.cc:507] Master 127.26.37.61:35057 requested a full tablet report, sending...
I20260809 07:49:06.262743 2158 ts_manager.cc:194] Registered new tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:34871)
I20260809 07:49:06.262809 2043 ts_manager.cc:194] Registered new tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:34871)
I20260809 07:49:06.264230 1926 ts_manager.cc:194] Registered new tserver with Master: 2f1a23ae20cb4950bac4989bc0c6506a (127.26.37.1:34871)
I20260809 07:49:06.266408 1926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:39411
W20260809 07:49:06.579100 2429 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:06.579406 2429 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:06.579589 2429 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:06.587864 2429 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:06.588071 2429 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:49:06.598004 2429 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:06.599835 2429 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:06.601776 2429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:06.613045 2444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:06.613099 2445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:06.614549 2447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:06.614717 2429 server_base.cc:1061] running on GCE node
I20260809 07:49:06.615357 2429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:06.616568 2429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:06.617928 2429 hybrid_clock.cc:648] HybridClock initialized: now 1786261746617745 us; error 139 us; skew 500 ppm
I20260809 07:49:06.618299 2429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:06.620548 2429 webserver.cc:533] Webserver started at http://127.26.37.2:36171/ using document root <none> and password file <none>
I20260809 07:49:06.621414 2429 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:06.621582 2429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:06.626974 2429 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 07:49:06.629662 2456 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:06.630985 2429 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:49:06.631165 2429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "af888a1cd1f8488a8f0d165a1fb0a12b"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:49:06.631889 2429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:06.651901 2429 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:06.652741 2429 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:06.653028 2429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:06.654006 2429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:06.655689 2429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:06.655798 2429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:06.655962 2429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:06.656035 2429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:06.691877 2429 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:34299
I20260809 07:49:06.691922 2622 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:34299 every 8 connection(s)
I20260809 07:49:06.693909 2429 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:49:06.698174 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2429
I20260809 07:49:06.698388 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:49:06.702351 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:49:06.713779 2624 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:49:06.714136 2624 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:06.714473 2625 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:49:06.714684 2625 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:06.715269 2624 heartbeater.cc:507] Master 127.26.37.62:34559 requested a full tablet report, sending...
I20260809 07:49:06.715312 2625 heartbeater.cc:507] Master 127.26.37.61:35057 requested a full tablet report, sending...
I20260809 07:49:06.716996 1926 ts_manager.cc:194] Registered new tserver with Master: af888a1cd1f8488a8f0d165a1fb0a12b (127.26.37.2:34299)
I20260809 07:49:06.717589 2043 ts_manager.cc:194] Registered new tserver with Master: af888a1cd1f8488a8f0d165a1fb0a12b (127.26.37.2:34299)
I20260809 07:49:06.718195 1926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:34961
I20260809 07:49:06.719316 2623 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36471
I20260809 07:49:06.719441 2623 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:06.720219 2623 heartbeater.cc:507] Master 127.26.37.60:36471 requested a full tablet report, sending...
I20260809 07:49:06.721194 2158 ts_manager.cc:194] Registered new tserver with Master: af888a1cd1f8488a8f0d165a1fb0a12b (127.26.37.2:34299)
I20260809 07:49:06.744972 2238 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "aa24f1d5861849978bc80d09961c6796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:06.745220 2238 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: LEADER
W20260809 07:49:07.048206 2634 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:07.048559 2634 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:07.048719 2634 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:07.056536 2634 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:07.056744 2634 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:49:07.066349 2634 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:07.067955 2634 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:07.069821 2634 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:07.079563 2645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:07.079640 2644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:07.080516 2647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:07.080906 2634 server_base.cc:1061] running on GCE node
I20260809 07:49:07.081502 2634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:07.082532 2634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:07.083775 2634 hybrid_clock.cc:648] HybridClock initialized: now 1786261747083625 us; error 120 us; skew 500 ppm
I20260809 07:49:07.084100 2634 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:07.090620 2634 webserver.cc:533] Webserver started at http://127.26.37.3:36903/ using document root <none> and password file <none>
I20260809 07:49:07.091558 2634 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:07.091737 2634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:07.098694 2634 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:49:07.101737 2656 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:07.103513 2634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:49:07.103689 2634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "b5b89a7e8b94446abeed2bc4318ffa32"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-bw88"
I20260809 07:49:07.104436 2634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:07.127563 2634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:07.128410 2634 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:07.128731 2634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:07.129746 2634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:07.131376 2634 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:07.131449 2634 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:07.131532 2634 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:07.131575 2634 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:07.176141 2634 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:33679
I20260809 07:49:07.176196 2806 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:33679 every 8 connection(s)
I20260809 07:49:07.177809 2634 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:07.182073 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2634
I20260809 07:49:07.182296 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:49:07.196220 2809 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:49:07.196266 2808 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:49:07.196673 2809 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:07.196669 2808 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:07.196906 2807 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36471
I20260809 07:49:07.197012 2807 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:07.197997 2807 heartbeater.cc:507] Master 127.26.37.60:36471 requested a full tablet report, sending...
I20260809 07:49:07.198045 2809 heartbeater.cc:507] Master 127.26.37.61:35057 requested a full tablet report, sending...
I20260809 07:49:07.198048 2808 heartbeater.cc:507] Master 127.26.37.62:34559 requested a full tablet report, sending...
I20260809 07:49:07.199978 2158 ts_manager.cc:194] Registered new tserver with Master: b5b89a7e8b94446abeed2bc4318ffa32 (127.26.37.3:33679)
I20260809 07:49:07.199973 1926 ts_manager.cc:194] Registered new tserver with Master: b5b89a7e8b94446abeed2bc4318ffa32 (127.26.37.3:33679)
I20260809 07:49:07.199986 2043 ts_manager.cc:194] Registered new tserver with Master: b5b89a7e8b94446abeed2bc4318ffa32 (127.26.37.3:33679)
I20260809 07:49:07.201025 1926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:59997
I20260809 07:49:07.208788 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:49:07.216287 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:49:07.216429 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1885
I20260809 07:49:07.217202 1970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:07.269202 2424 heartbeater.cc:499] Master 127.26.37.62:34559 was elected leader, sending a full tablet report...
I20260809 07:49:07.323341 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1885
I20260809 07:49:07.353806 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34559
--webserver_interface=127.26.37.62
--webserver_port=32775
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471 with env {}
W20260809 07:49:07.691694 2818 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:07.692083 2818 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:07.692205 2818 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:07.700968 2818 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:07.701063 2818 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:07.701182 2818 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:07.701267 2818 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:07.711402 2818 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=32775
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:07.713068 2818 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:07.715126 2818 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:49:07.721072 2624 heartbeater.cc:499] Master 127.26.37.62:34559 was elected leader, sending a full tablet report...
W20260809 07:49:07.723573 2624 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:34559 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:34559: connect: Connection refused (error 111)
W20260809 07:49:07.724478 2825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:07.724541 2824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:07.726468 2829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:07.726917 2818 server_base.cc:1061] running on GCE node
I20260809 07:49:07.727643 2818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:07.729235 2818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:07.730473 2818 hybrid_clock.cc:648] HybridClock initialized: now 1786261747730314 us; error 131 us; skew 500 ppm
I20260809 07:49:07.730844 2818 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:07.732959 2818 webserver.cc:533] Webserver started at http://127.26.37.62:32775/ using document root <none> and password file <none>
I20260809 07:49:07.733821 2818 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:07.733964 2818 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:07.740162 2818 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:49:07.743209 2835 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:07.744745 2818 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:49:07.744941 2818 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "aa24f1d5861849978bc80d09961c6796"
format_stamp: "Formatted at 2026-08-09 07:48:57 on dist-test-slave-bw88"
I20260809 07:49:07.745802 2818 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:07.758821 2818 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:07.759619 2818 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:07.759934 2818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:07.780293 2818 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34559
I20260809 07:49:07.780313 2904 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34559 every 8 connection(s)
I20260809 07:49:07.781941 2818 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:07.784550 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2818
I20260809 07:49:07.784873 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2244
I20260809 07:49:07.790156 2905 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:07.793996 2905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap starting.
I20260809 07:49:07.794373 2418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:07.812017 2905 log.cc:826] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:07.820557 2905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:07.821067 2905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: Bootstrap complete.
I20260809 07:49:07.827805 2905 raft_consensus.cc:359] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:07.828495 2905 raft_consensus.cc:740] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa24f1d5861849978bc80d09961c6796, State: Initialized, Role: FOLLOWER
I20260809 07:49:07.829422 2905 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:07.830299 2913 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:07.830579 2913 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:07.831306 2905 sys_catalog.cc:565] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:07.841754 2928 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:49:07.841928 2928 catalog_manager.cc:1588] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: loading cluster ID for follower catalog manager: success
I20260809 07:49:07.846861 2928 catalog_manager.cc:1610] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: acquiring CA information for follower catalog manager: success
I20260809 07:49:07.849743 2928 catalog_manager.cc:1638] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:07.879364 2818 master_runner.cc:264] No leader master found from master aa24f1d5861849978bc80d09961c6796
I20260809 07:49:07.879528 2818 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:49:07.905409 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2244
I20260809 07:49:07.939581 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34871
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--webserver_port=34659
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:49:08.203143 2808 heartbeater.cc:499] Master 127.26.37.62:34559 was elected leader, sending a full tablet report...
I20260809 07:49:08.214341 2860 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b5b89a7e8b94446abeed2bc4318ffa32" instance_seqno: 1786261747166435) as {username='slave'} at 127.26.37.3:59055; Asking this server to re-register.
I20260809 07:49:08.215332 2808 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:08.217319 2860 ts_manager.cc:194] Registered new tserver with Master: b5b89a7e8b94446abeed2bc4318ffa32 (127.26.37.3:33679)
W20260809 07:49:08.293790 2937 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:08.294231 2937 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:08.294390 2937 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:08.302675 2937 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:08.302853 2937 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:08.312309 2937 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34871
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34659
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:08.314030 2937 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:08.315874 2937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:08.325708 2944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:08.325655 2946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:08.326850 2950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:08.326840 2937 server_base.cc:1061] running on GCE node
I20260809 07:49:08.327491 2937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:08.328567 2937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:08.329887 2937 hybrid_clock.cc:648] HybridClock initialized: now 1786261748329730 us; error 126 us; skew 500 ppm
I20260809 07:49:08.330209 2937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:08.332346 2937 webserver.cc:533] Webserver started at http://127.26.37.1:34659/ using document root <none> and password file <none>
I20260809 07:49:08.333130 2937 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:08.333273 2937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:08.339102 2937 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:49:08.341974 2959 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:08.343346 2937 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:49:08.343559 2937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:49:08.344254 2937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:08.363797 2937 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:08.364658 2937 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:08.364971 2937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:08.365948 2937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:08.367724 2937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:08.367832 2937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:08.367995 2937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:08.368077 2937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:08.413407 2937 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34871
I20260809 07:49:08.413473 3112 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34871 every 8 connection(s)
I20260809 07:49:08.415032 2937 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:08.419363 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2937
I20260809 07:49:08.419711 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2937
I20260809 07:49:08.452186 3105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:08.476469 3130 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aa24f1d5861849978bc80d09961c6796)
I20260809 07:49:08.476781 3130 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:08.478951 3130 leader_election.cc:290] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aa24f1d5861849978bc80d09961c6796 (127.26.37.62:34559), 0b75351d11604db78444ee9c04118d04 (127.26.37.60:36471)
I20260809 07:49:08.494601 2174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0b75351d11604db78444ee9c04118d04" is_pre_election: true
I20260809 07:49:08.495659 2871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aa24f1d5861849978bc80d09961c6796" is_pre_election: true
I20260809 07:49:08.496176 2871 raft_consensus.cc:2468] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee8bafe05bd540e0a9a1d488adabdee8 in term 2.
I20260809 07:49:08.496831 2027 leader_election.cc:304] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa24f1d5861849978bc80d09961c6796, ee8bafe05bd540e0a9a1d488adabdee8; no voters: 0b75351d11604db78444ee9c04118d04
I20260809 07:49:08.497251 3130 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:49:08.497471 3130 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader aa24f1d5861849978bc80d09961c6796)
I20260809 07:49:08.497630 3130 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:49:08.499934 3130 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:08.500618 3130 leader_election.cc:290] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [CANDIDATE]: Term 3 election: Requested vote from peers aa24f1d5861849978bc80d09961c6796 (127.26.37.62:34559), 0b75351d11604db78444ee9c04118d04 (127.26.37.60:36471)
I20260809 07:49:08.501139 2871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aa24f1d5861849978bc80d09961c6796"
I20260809 07:49:08.501385 2871 raft_consensus.cc:3060] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:49:08.501389 2174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0b75351d11604db78444ee9c04118d04"
I20260809 07:49:08.505443 2871 raft_consensus.cc:2468] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee8bafe05bd540e0a9a1d488adabdee8 in term 3.
I20260809 07:49:08.505900 2027 leader_election.cc:304] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa24f1d5861849978bc80d09961c6796, ee8bafe05bd540e0a9a1d488adabdee8; no voters: 0b75351d11604db78444ee9c04118d04
I20260809 07:49:08.506217 3130 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:49:08.506809 3130 raft_consensus.cc:697] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [term 3 LEADER]: Becoming Leader. State: Replica: ee8bafe05bd540e0a9a1d488adabdee8, State: Running, Role: LEADER
I20260809 07:49:08.507282 3130 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:08.509148 3135 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee8bafe05bd540e0a9a1d488adabdee8. Latest consensus state: current_term: 3 leader_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:08.509420 3135 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:08.510335 3137 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:49:08.512887 3137 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:49:08.513908 3137 catalog_manager.cc:1295] Loaded cluster ID: c44bcd243c5a4d8f8a99eb223a4b4a6e
I20260809 07:49:08.514003 3137 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:49:08.515426 3137 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:49:08.516446 3137 catalog_manager.cc:6103] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8: Loaded TSK: 0
I20260809 07:49:08.517565 3137 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260809 07:49:08.552548 2966 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.26.37.60:36471, user_credentials={real_user=slave}} blocked reactor thread for 97811.3us
I20260809 07:49:08.554059 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2937
W20260809 07:49:08.578022 2837 connection.cc:573] server connection from 127.26.37.1:59793 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:49:08.578789 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34871
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--webserver_port=34659
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260809 07:49:08.651126 3139 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aa24f1d5861849978bc80d09961c6796)
I20260809 07:49:08.651445 3139 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } }
I20260809 07:49:08.653195 3139 leader_election.cc:290] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aa24f1d5861849978bc80d09961c6796 (127.26.37.62:34559), ee8bafe05bd540e0a9a1d488adabdee8 (127.26.37.61:35057)
I20260809 07:49:08.653942 2054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b75351d11604db78444ee9c04118d04" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" is_pre_election: true
I20260809 07:49:08.660629 2871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b75351d11604db78444ee9c04118d04" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aa24f1d5861849978bc80d09961c6796" is_pre_election: true
I20260809 07:49:08.660976 2871 raft_consensus.cc:2393] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0b75351d11604db78444ee9c04118d04 in current term 3: Already voted for candidate ee8bafe05bd540e0a9a1d488adabdee8 in this term.
I20260809 07:49:08.661684 2139 leader_election.cc:304] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b75351d11604db78444ee9c04118d04; no voters: aa24f1d5861849978bc80d09961c6796, ee8bafe05bd540e0a9a1d488adabdee8
I20260809 07:49:08.662103 3139 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:49:08.665148 3139 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:49:08.732076 2624 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34559
I20260809 07:49:08.732328 2624 heartbeater.cc:499] Master 127.26.37.62:34559 was elected leader, sending a full tablet report...
I20260809 07:49:08.733255 2860 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "af888a1cd1f8488a8f0d165a1fb0a12b" instance_seqno: 1786261746683449) as {username='slave'} at 127.26.37.2:52271; Asking this server to re-register.
I20260809 07:49:08.733883 2624 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:08.735088 2860 ts_manager.cc:194] Registered new tserver with Master: af888a1cd1f8488a8f0d165a1fb0a12b (127.26.37.2:34299)
W20260809 07:49:08.932479 3138 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:08.932796 3138 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:08.932947 3138 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:49:08.933037 3138 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:49:08.937065 2871 raft_consensus.cc:1275] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [term 3 FOLLOWER]: Refusing update from remote peer ee8bafe05bd540e0a9a1d488adabdee8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:49:08.937992 3135 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:49:08.942303 3146 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee8bafe05bd540e0a9a1d488adabdee8. Latest consensus state: current_term: 3 leader_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
W20260809 07:49:08.943203 3138 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:08.943480 3138 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:08.943696 3130 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:08.943960 3130 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:08.944571 2174 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [term 3 FOLLOWER]: Refusing update from remote peer ee8bafe05bd540e0a9a1d488adabdee8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:49:08.944962 3146 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:08.945214 3135 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:49:08.951202 3146 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:08.951472 3146 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa24f1d5861849978bc80d09961c6796 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:08.952512 3139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee8bafe05bd540e0a9a1d488adabdee8. Latest consensus state: current_term: 3 leader_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:08.952468 3152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:08.952793 3139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:08.952864 3152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b75351d11604db78444ee9c04118d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:08.962224 3138 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34871
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34659
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:08.964444 3138 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:08.966349 3138 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:08.975776 3158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:08.975769 3159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:08.977635 3161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:08.978076 3138 server_base.cc:1061] running on GCE node
I20260809 07:49:08.978780 3138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:08.979924 3138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:08.981221 3138 hybrid_clock.cc:648] HybridClock initialized: now 1786261748981055 us; error 86 us; skew 500 ppm
I20260809 07:49:08.981583 3138 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:08.983879 3138 webserver.cc:533] Webserver started at http://127.26.37.1:34659/ using document root <none> and password file <none>
I20260809 07:49:08.984650 3138 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:08.984807 3138 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:08.990568 3138 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 07:49:08.993439 3168 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:08.995024 3138 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:49:08.995213 3138 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:49:08.995950 3138 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:09.016640 3138 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:09.017554 3138 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:09.017833 3138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:09.018764 3138 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:09.020371 3138 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:09.020488 3138 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:09.020614 3138 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:09.020699 3138 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:49:09.021054 3138 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:49:09.021183 3138 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:49:09.021296 3138 fault_injection.cc:59] injecting fault for kudu (pid 3138): FLAGS_fault_before_start (process will exit)
I20260809 07:49:09.042126 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34871
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--webserver_port=34659
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:49:09.394080 3178 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:09.394382 3178 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:09.394480 3178 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:09.402702 3178 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:09.402889 3178 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:09.412501 3178 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39125,127.26.37.21:39189,127.26.37.22:35445,127.26.37.23:33193,127.26.37.24:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34871
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34659
--tserver_master_addrs=127.26.37.62:34559,127.26.37.61:35057,127.26.37.60:36471
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:09.414079 3178 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:09.415988 3178 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:09.425756 3185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:09.425786 3186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:09.427100 3189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:09.427500 3178 server_base.cc:1061] running on GCE node
I20260809 07:49:09.428133 3178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:09.429244 3178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:09.430567 3178 hybrid_clock.cc:648] HybridClock initialized: now 1786261749430373 us; error 155 us; skew 500 ppm
I20260809 07:49:09.430922 3178 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:09.433526 3178 webserver.cc:533] Webserver started at http://127.26.37.1:34659/ using document root <none> and password file <none>
I20260809 07:49:09.434315 3178 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:09.434513 3178 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:09.440496 3178 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260809 07:49:09.443388 3198 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:09.444891 3178 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 07:49:09.445039 3178 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2f1a23ae20cb4950bac4989bc0c6506a"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-bw88"
I20260809 07:49:09.445760 3178 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:09.448532 3135 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa24f1d5861849978bc80d09961c6796" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34559 } } peers { permanent_uuid: "ee8bafe05bd540e0a9a1d488adabdee8" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35057 } } peers { permanent_uuid: "0b75351d11604db78444ee9c04118d04" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36471 } } }
I20260809 07:49:09.448757 3135 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8bafe05bd540e0a9a1d488adabdee8 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:09.463730 3178 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:09.464541 3178 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:09.464845 3178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:09.465843 3178 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:09.467442 3178 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:09.467551 3178 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:09.467703 3178 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:09.467779 3178 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:09.504267 3178 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34871
I20260809 07:49:09.504310 3339 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34871 every 8 connection(s)
I20260809 07:49:09.505959 3178 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:09.513726 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3178
I20260809 07:49:09.514163 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3178
I20260809 07:49:09.547326 3334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:09.631577 3344 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35057
I20260809 07:49:09.633404 3344 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:09.633528 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3178
I20260809 07:49:09.657915 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2429
W20260809 07:49:09.658773 3351 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:49:09.509515 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:57253 (local address 127.26.37.60:36471)
0809 07:49:09.522375 (+ 12860us) server_negotiation.cc:207] Beginning negotiation
0809 07:49:09.522384 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:49:09.529858 (+ 7474us) server_negotiation.cc:408] Connection header received
0809 07:49:09.534432 (+ 4574us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:49:09.534444 (+ 12us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:49:09.534552 (+ 108us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:49:09.534668 (+ 116us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:49:09.536478 (+ 1810us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.538340 (+ 1862us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.542581 (+ 4241us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.543214 (+ 633us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:49:09.657793 (+114579us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:57253: BlockingRecv error: recv got EOF from 127.26.37.1:57253 (error 108)
Metrics: {"server-negotiator.queue_time_us":12688,"thread_start_us":12840,"threads_started":1}
W20260809 07:49:09.659373 3357 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:49:09.530046 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:36409 (local address 127.26.37.62:34559)
0809 07:49:09.548737 (+ 18691us) server_negotiation.cc:207] Beginning negotiation
0809 07:49:09.548743 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:49:09.548765 (+ 22us) server_negotiation.cc:408] Connection header received
0809 07:49:09.548851 (+ 86us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:49:09.548860 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:49:09.548946 (+ 86us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:49:09.549073 (+ 127us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:49:09.633864 (+ 84791us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.637000 (+ 3136us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.657731 (+ 20731us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:36409: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":18560,"thread_start_us":18469,"threads_started":1}
I20260809 07:49:09.667340 2618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:09.751861 2625 heartbeater.cc:499] Master 127.26.37.61:35057 was elected leader, sending a full tablet report...
I20260809 07:49:09.754366 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2429
I20260809 07:49:09.779842 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2634
I20260809 07:49:09.789361 2802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:09.876566 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2634
I20260809 07:49:09.904768 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2818
I20260809 07:49:09.911187 2899 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:10.017020 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2818
I20260809 07:49:10.042749 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1977
I20260809 07:49:10.043895 2077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:10.190096 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1977
I20260809 07:49:10.222329 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2090
I20260809 07:49:10.223490 2199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:10.332597 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2090
2026-08-09T07:49:10Z chronyd exiting
2026-08-09T07:49:10Z chronyd exiting
2026-08-09T07:49:10Z chronyd exiting
2026-08-09T07:49:10Z chronyd exiting
2026-08-09T07:49:10Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/2 (13597 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/3
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:49:17: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:49:26.584133 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39893
--webserver_interface=127.26.37.62
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683 with env {}
W20260809 07:49:26.920905 3741 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:26.921283 3741 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:26.921456 3741 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:26.929384 3741 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:26.929497 3741 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:26.929555 3741 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:26.929603 3741 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:26.939320 3741 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:26.940954 3741 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:26.942806 3741 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:26.950656 3748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:26.950851 3749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:26.951812 3751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:26.952804 3741 server_base.cc:1061] running on GCE node
I20260809 07:49:26.953660 3741 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:26.953864 3741 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:26.953969 3741 hybrid_clock.cc:648] HybridClock initialized: now 1786261766953922 us; error 0 us; skew 500 ppm
I20260809 07:49:26.954319 3741 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:26.956979 3741 webserver.cc:533] Webserver started at http://127.26.37.62:43915/ using document root <none> and password file <none>
I20260809 07:49:26.957744 3741 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:26.957867 3741 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:26.958236 3741 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:26.961386 3741 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "93f500acf72b4a6fb74061968b61acd7"
format_stamp: "Formatted at 2026-08-09 07:49:26 on dist-test-slave-bw88"
I20260809 07:49:26.962478 3741 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "93f500acf72b4a6fb74061968b61acd7"
format_stamp: "Formatted at 2026-08-09 07:49:26 on dist-test-slave-bw88"
I20260809 07:49:26.967689 3741 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:49:26.971055 3756 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:26.972508 3741 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:49:26.972731 3741 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "93f500acf72b4a6fb74061968b61acd7"
format_stamp: "Formatted at 2026-08-09 07:49:26 on dist-test-slave-bw88"
I20260809 07:49:26.972980 3741 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:27.003984 3741 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:27.005084 3741 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:27.005479 3741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:27.026880 3741 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39893
I20260809 07:49:27.026899 3827 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39893 every 8 connection(s)
I20260809 07:49:27.028462 3741 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:27.032809 3828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:27.036665 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3741
I20260809 07:49:27.036881 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:49:27.038273 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39555
--webserver_interface=127.26.37.61
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683 with env {}
I20260809 07:49:27.040248 3828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:49:27.061892 3828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:49:27.063090 3761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39555: connect: Connection refused (error 111)
W20260809 07:49:27.064903 3828 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39555: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39555: connect: Connection refused (error 111)
I20260809 07:49:27.114281 3828 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } attempt: 1
W20260809 07:49:27.116782 3828 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39555: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39555: connect: Connection refused (error 111)
I20260809 07:49:27.185065 3828 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } attempt: 2
W20260809 07:49:27.187439 3828 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39555: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39555: connect: Connection refused (error 111)
I20260809 07:49:27.278759 3828 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } attempt: 3
W20260809 07:49:27.281013 3828 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39555: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39555: connect: Connection refused (error 111)
W20260809 07:49:27.401348 3833 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:27.401732 3833 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:27.401897 3833 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:27.410259 3833 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:27.410382 3833 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:27.410521 3833 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:27.410609 3833 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:27.420730 3833 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:27.422533 3833 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:27.424311 3828 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } attempt: 4
I20260809 07:49:27.424500 3833 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:27.426720 3828 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39555: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39555: connect: Connection refused (error 111)
W20260809 07:49:27.433939 3848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:27.434481 3847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:27.435056 3851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:27.435855 3833 server_base.cc:1061] running on GCE node
I20260809 07:49:27.436722 3833 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:27.436896 3833 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:27.436973 3833 hybrid_clock.cc:648] HybridClock initialized: now 1786261767436954 us; error 0 us; skew 500 ppm
I20260809 07:49:27.437286 3833 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:27.441306 3833 webserver.cc:533] Webserver started at http://127.26.37.61:40637/ using document root <none> and password file <none>
I20260809 07:49:27.442212 3833 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:27.442322 3833 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:27.442679 3833 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:27.447024 3833 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "a8574a2832484382b63407f07c2a72ae"
format_stamp: "Formatted at 2026-08-09 07:49:27 on dist-test-slave-bw88"
I20260809 07:49:27.447885 3833 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "a8574a2832484382b63407f07c2a72ae"
format_stamp: "Formatted at 2026-08-09 07:49:27 on dist-test-slave-bw88"
I20260809 07:49:27.453924 3833 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260809 07:49:27.457727 3856 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:27.459206 3833 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:49:27.459419 3833 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "a8574a2832484382b63407f07c2a72ae"
format_stamp: "Formatted at 2026-08-09 07:49:27 on dist-test-slave-bw88"
I20260809 07:49:27.459657 3833 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:27.475620 3833 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:27.476639 3833 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:27.476955 3833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:27.499538 3833 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:39555
I20260809 07:49:27.499577 3926 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:39555 every 8 connection(s)
I20260809 07:49:27.501191 3833 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:49:27.502960 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3833
I20260809 07:49:27.503152 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:49:27.504619 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37683
--webserver_interface=127.26.37.60
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683 with env {}
I20260809 07:49:27.506989 3927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:27.515376 3927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:49:27.530757 3927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:49:27.539884 3927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:49:27.541002 3859 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
W20260809 07:49:27.542730 3927 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
I20260809 07:49:27.592034 3927 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } attempt: 1
W20260809 07:49:27.594224 3927 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
I20260809 07:49:27.662510 3927 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } attempt: 2
W20260809 07:49:27.664824 3927 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
I20260809 07:49:27.725996 3828 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } attempt: 5
I20260809 07:49:27.733160 3828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:49:27.735453 3828 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
I20260809 07:49:27.756100 3927 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } attempt: 3
W20260809 07:49:27.758219 3927 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
I20260809 07:49:27.786693 3828 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } attempt: 1
W20260809 07:49:27.788959 3828 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
I20260809 07:49:27.857198 3828 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } attempt: 2
W20260809 07:49:27.859413 3828 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
W20260809 07:49:27.862435 3929 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:27.862803 3929 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:27.862967 3929 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:27.871814 3929 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:27.871937 3929 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:27.872043 3929 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:27.872155 3929 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:27.882766 3929 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:27.884604 3929 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:27.886659 3929 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:27.895399 3941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:27.895455 3940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:27.897068 3944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:27.897296 3929 server_base.cc:1061] running on GCE node
I20260809 07:49:27.898136 3929 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:27.898283 3929 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:27.898347 3929 hybrid_clock.cc:648] HybridClock initialized: now 1786261767898329 us; error 0 us; skew 500 ppm
I20260809 07:49:27.898747 3929 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:27.901467 3927 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } attempt: 4
I20260809 07:49:27.901597 3929 webserver.cc:533] Webserver started at http://127.26.37.60:37995/ using document root <none> and password file <none>
I20260809 07:49:27.902573 3929 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:27.902714 3929 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:27.903194 3929 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
W20260809 07:49:27.903688 3927 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111)
I20260809 07:49:27.905972 3929 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4"
format_stamp: "Formatted at 2026-08-09 07:49:27 on dist-test-slave-bw88"
I20260809 07:49:27.906782 3929 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4"
format_stamp: "Formatted at 2026-08-09 07:49:27 on dist-test-slave-bw88"
I20260809 07:49:27.912091 3929 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:49:27.915524 3951 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:27.916965 3929 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:49:27.917160 3929 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4"
format_stamp: "Formatted at 2026-08-09 07:49:27 on dist-test-slave-bw88"
I20260809 07:49:27.917503 3929 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:27.934882 3929 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:27.935797 3929 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:27.936072 3929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:27.957576 3929 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:37683
I20260809 07:49:27.957527 4023 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:37683 every 8 connection(s)
I20260809 07:49:27.959108 3929 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:49:27.959748 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3929
I20260809 07:49:27.960009 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:49:27.965061 4025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:27.966447 3828 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } attempt: 3
I20260809 07:49:27.974906 4025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:49:27.987221 4025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:49:27.987666 3828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap starting.
I20260809 07:49:27.991266 3828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:27.992584 3828 log.cc:826] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:27.995618 3828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: No bootstrap required, opened a new log
I20260809 07:49:27.998113 4025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:49:28.003015 3828 raft_consensus.cc:359] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.003433 3828 raft_consensus.cc:385] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:28.003592 3828 raft_consensus.cc:740] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f500acf72b4a6fb74061968b61acd7, State: Initialized, Role: FOLLOWER
I20260809 07:49:28.004418 3828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.005586 4034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.005949 4034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:28.006703 3828 sys_catalog.cc:565] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:49:28.008318 3764 tablet.cc:2406] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:49:28.009449 4025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Bootstrap starting.
I20260809 07:49:28.013643 4025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:28.014868 4025 log.cc:826] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:28.017848 4025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: No bootstrap required, opened a new log
W20260809 07:49:28.022251 4050 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:49:28.022372 4050 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:49:28.025686 4025 raft_consensus.cc:359] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.026077 4025 raft_consensus.cc:385] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:28.026213 4025 raft_consensus.cc:740] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74e2a4f17ac4fd7b32e1d55390fabd4, State: Initialized, Role: FOLLOWER
I20260809 07:49:28.027062 4025 consensus_queue.cc:260] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.021771 3741 master_runner.cc:187] Error getting master registration for 127.26.37.61:39555: OK, instance_id { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" instance_seqno: 1786261767493563 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:49:28.028828 3741 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:49:28.029471 4052 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.029793 4052 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:28.030869 4025 sys_catalog.cc:565] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:49:28.048250 4067 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:49:28.048408 4067 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:49:28.050079 3929 master_runner.cc:187] Error getting master registration for 127.26.37.61:39555: OK, instance_id { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" instance_seqno: 1786261767493563 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:49:28.056941 3929 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:49:28.188359 3959 tablet.cc:2406] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:49:28.202973 3927 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } attempt: 5
I20260809 07:49:28.216804 3927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: Bootstrap starting.
I20260809 07:49:28.221397 3927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:28.223140 3927 log.cc:826] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:28.230707 3927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: No bootstrap required, opened a new log
I20260809 07:49:28.239044 3927 raft_consensus.cc:359] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.239391 3927 raft_consensus.cc:385] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:28.239523 3927 raft_consensus.cc:740] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8574a2832484382b63407f07c2a72ae, State: Initialized, Role: FOLLOWER
I20260809 07:49:28.239558 4034 raft_consensus.cc:493] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:49:28.239888 4034 raft_consensus.cc:515] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.240218 3927 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.241811 4034 leader_election.cc:290] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8574a2832484382b63407f07c2a72ae (127.26.37.61:39555), d74e2a4f17ac4fd7b32e1d55390fabd4 (127.26.37.60:37683)
I20260809 07:49:28.243578 3988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93f500acf72b4a6fb74061968b61acd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" is_pre_election: true
I20260809 07:49:28.244205 3988 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f500acf72b4a6fb74061968b61acd7 in term 0.
I20260809 07:49:28.244877 3761 leader_election.cc:304] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93f500acf72b4a6fb74061968b61acd7, d74e2a4f17ac4fd7b32e1d55390fabd4; no voters:
I20260809 07:49:28.245352 4034 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:49:28.245509 4034 raft_consensus.cc:493] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:49:28.245615 4034 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:28.247262 4075 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.247540 4075 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:28.248940 4034 raft_consensus.cc:515] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.249279 3927 sys_catalog.cc:565] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:28.250030 3888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93f500acf72b4a6fb74061968b61acd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8574a2832484382b63407f07c2a72ae"
I20260809 07:49:28.250464 3888 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:28.253958 3888 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f500acf72b4a6fb74061968b61acd7 in term 1.
I20260809 07:49:28.254386 3889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93f500acf72b4a6fb74061968b61acd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8574a2832484382b63407f07c2a72ae" is_pre_election: true
I20260809 07:49:28.254658 3889 raft_consensus.cc:2376] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 93f500acf72b4a6fb74061968b61acd7 in term 1. Re-sending same reply.
I20260809 07:49:28.255111 3761 leader_election.cc:304] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93f500acf72b4a6fb74061968b61acd7, a8574a2832484382b63407f07c2a72ae; no voters:
I20260809 07:49:28.257908 4034 leader_election.cc:290] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [CANDIDATE]: Term 1 election: Requested vote from peers a8574a2832484382b63407f07c2a72ae (127.26.37.61:39555), d74e2a4f17ac4fd7b32e1d55390fabd4 (127.26.37.60:37683)
I20260809 07:49:28.258145 4034 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:49:28.258285 4034 raft_consensus.cc:697] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 1 LEADER]: Becoming Leader. State: Replica: 93f500acf72b4a6fb74061968b61acd7, State: Running, Role: LEADER
I20260809 07:49:28.258782 4034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:28.265316 4034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93f500acf72b4a6fb74061968b61acd7. Latest consensus state: current_term: 1 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.265628 4034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:28.272646 3988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93f500acf72b4a6fb74061968b61acd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4"
I20260809 07:49:28.272881 3988 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:28.276314 3988 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f500acf72b4a6fb74061968b61acd7 in term 1.
I20260809 07:49:28.282052 4082 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:49:28.285033 4082 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:49:28.300863 3988 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 1 FOLLOWER]: Refusing update from remote peer 93f500acf72b4a6fb74061968b61acd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:49:28.302131 4034 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:49:28.309846 3888 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 1 FOLLOWER]: Refusing update from remote peer 93f500acf72b4a6fb74061968b61acd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:49:28.311054 4034 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:49:28.329562 4052 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93f500acf72b4a6fb74061968b61acd7. Latest consensus state: current_term: 1 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.329867 4052 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:28.333282 4087 mvcc.cc:204] Tried to move back new op lower bound from 7316528202949210112 to 7316528202802176000. Current Snapshot: MvccSnapshot[applied={T|T < 7316528202949210112}]
I20260809 07:49:28.333268 4075 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93f500acf72b4a6fb74061968b61acd7. Latest consensus state: current_term: 1 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.333547 4075 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:28.337405 4092 mvcc.cc:204] Tried to move back new op lower bound from 7316528202949210112 to 7316528202802176000. Current Snapshot: MvccSnapshot[applied={T|T < 7316528202949210112}]
I20260809 07:49:28.351366 4052 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.351572 4052 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:28.355767 4082 catalog_manager.cc:1383] Generated new cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:28.355895 4082 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:49:28.359774 4034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.360030 4034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:28.360373 4034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.360653 4034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:28.362133 4075 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:28.362362 4075 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:28.380720 4102 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:28.380841 4102 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: loading cluster ID for follower catalog manager: success
I20260809 07:49:28.381071 4082 catalog_manager.cc:1406] Generated new certificate authority record
W20260809 07:49:28.382166 4102 catalog_manager.cc:1612] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260809 07:49:28.382301 4102 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:49:28.383441 4082 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:49:28.411365 4082 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Generated new TSK 0
I20260809 07:49:28.412668 4082 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:49:28.423588 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--time_source=system_unsync with env {}
W20260809 07:49:28.814878 4112 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:28.815222 4112 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:28.815356 4112 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:28.823896 4112 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:28.824091 4112 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:28.834105 4112 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:28.835794 4112 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:28.837731 4112 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:28.847285 4117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:28.847364 4118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:28.854606 4120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:28.855039 4112 server_base.cc:1061] running on GCE node
I20260809 07:49:28.855648 4112 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:28.855829 4112 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:28.855911 4112 hybrid_clock.cc:648] HybridClock initialized: now 1786261768855891 us; error 0 us; skew 500 ppm
I20260809 07:49:28.856233 4112 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:28.858875 4112 webserver.cc:533] Webserver started at http://127.26.37.1:38913/ using document root <none> and password file <none>
I20260809 07:49:28.859822 4112 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:28.859943 4112 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:28.860354 4112 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:28.863749 4112 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:28.864740 4112 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:28.871321 4112 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260809 07:49:28.874691 4129 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:28.876000 4112 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:49:28.876197 4112 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:28.876416 4112 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:28.900399 4112 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:28.901324 4112 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:28.901677 4112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:28.902645 4112 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:28.904284 4112 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:28.904420 4112 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:28.904574 4112 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:28.904671 4112 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:28.939826 4112 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43731
I20260809 07:49:28.939875 4284 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43731 every 8 connection(s)
I20260809 07:49:28.941612 4112 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:28.952023 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4112
I20260809 07:49:28.952232 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:49:28.957038 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--time_source=system_unsync with env {}
I20260809 07:49:28.959887 4286 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:28.959966 4285 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:28.960284 4286 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:28.961277 4286 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:28.960289 4285 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:28.962808 4285 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:28.963781 3974 ts_manager.cc:194] Registered new tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
I20260809 07:49:28.965902 3777 ts_manager.cc:194] Registered new tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
I20260809 07:49:28.968149 3777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:39143
I20260809 07:49:28.970537 4287 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:28.970651 4287 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:28.971103 4287 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:28.972849 3875 ts_manager.cc:194] Registered new tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
I20260809 07:49:29.050309 4067 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:29.050428 4067 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: loading cluster ID for follower catalog manager: success
I20260809 07:49:29.053095 4067 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: acquiring CA information for follower catalog manager: success
I20260809 07:49:29.054769 4067 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:49:29.308636 4297 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:29.308962 4297 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:29.309108 4297 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:29.317260 4297 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:29.317519 4297 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:49:29.327325 4297 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:29.328902 4297 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:29.330749 4297 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:29.340266 4306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:29.340224 4305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:29.341131 4309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:29.341867 4297 server_base.cc:1061] running on GCE node
I20260809 07:49:29.342496 4297 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:29.342689 4297 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:29.342792 4297 hybrid_clock.cc:648] HybridClock initialized: now 1786261769342754 us; error 0 us; skew 500 ppm
I20260809 07:49:29.343072 4297 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:29.345094 4297 webserver.cc:533] Webserver started at http://127.26.37.2:35387/ using document root <none> and password file <none>
I20260809 07:49:29.345914 4297 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:29.346047 4297 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:29.346424 4297 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:29.348690 4297 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "dbbd7c27adec465ba9108196b61f7a3c"
format_stamp: "Formatted at 2026-08-09 07:49:29 on dist-test-slave-bw88"
I20260809 07:49:29.349587 4297 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "dbbd7c27adec465ba9108196b61f7a3c"
format_stamp: "Formatted at 2026-08-09 07:49:29 on dist-test-slave-bw88"
I20260809 07:49:29.354717 4297 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 07:49:29.357878 4315 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:29.359112 4297 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:29.359287 4297 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "dbbd7c27adec465ba9108196b61f7a3c"
format_stamp: "Formatted at 2026-08-09 07:49:29 on dist-test-slave-bw88"
I20260809 07:49:29.359490 4297 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:29.375325 4297 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:29.376133 4297 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:29.376415 4297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:29.377314 4297 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:29.379055 4297 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:29.379168 4297 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:29.379303 4297 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:29.379407 4297 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:29.385989 4102 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: acquiring CA information for follower catalog manager: success
I20260809 07:49:29.388494 4102 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:29.414445 4297 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:45817
I20260809 07:49:29.414552 4468 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:45817 every 8 connection(s)
I20260809 07:49:29.416447 4297 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:49:29.423905 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4297
I20260809 07:49:29.424121 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:49:29.428824 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--time_source=system_unsync with env {}
I20260809 07:49:29.435604 4472 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:29.435684 4475 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:29.435724 4473 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:29.436100 4473 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:29.436096 4475 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:29.437134 4475 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:29.437163 4473 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:29.436096 4472 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:29.438549 4472 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:29.438983 3875 ts_manager.cc:194] Registered new tserver with Master: dbbd7c27adec465ba9108196b61f7a3c (127.26.37.2:45817)
I20260809 07:49:29.439128 3974 ts_manager.cc:194] Registered new tserver with Master: dbbd7c27adec465ba9108196b61f7a3c (127.26.37.2:45817)
I20260809 07:49:29.439563 3777 ts_manager.cc:194] Registered new tserver with Master: dbbd7c27adec465ba9108196b61f7a3c (127.26.37.2:45817)
I20260809 07:49:29.440366 3777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:56967
W20260809 07:49:29.783305 4481 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:29.783648 4481 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:29.783774 4481 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:29.792579 4481 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:29.792796 4481 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:49:29.802464 4481 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:29.804114 4481 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:29.806181 4481 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:29.815443 4489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:29.815536 4490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:29.816135 4493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:29.817644 4481 server_base.cc:1061] running on GCE node
I20260809 07:49:29.818217 4481 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:29.818397 4481 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:29.818503 4481 hybrid_clock.cc:648] HybridClock initialized: now 1786261769818459 us; error 0 us; skew 500 ppm
I20260809 07:49:29.818847 4481 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:29.821095 4481 webserver.cc:533] Webserver started at http://127.26.37.3:40539/ using document root <none> and password file <none>
I20260809 07:49:29.821923 4481 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:29.822068 4481 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:29.822456 4481 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:29.824921 4481 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "382ede2e856d4df8b6da244c62cd2c11"
format_stamp: "Formatted at 2026-08-09 07:49:29 on dist-test-slave-bw88"
I20260809 07:49:29.825793 4481 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "382ede2e856d4df8b6da244c62cd2c11"
format_stamp: "Formatted at 2026-08-09 07:49:29 on dist-test-slave-bw88"
I20260809 07:49:29.832254 4481 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 07:49:29.835855 4500 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:29.837293 4481 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:49:29.837538 4481 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "382ede2e856d4df8b6da244c62cd2c11"
format_stamp: "Formatted at 2026-08-09 07:49:29 on dist-test-slave-bw88"
I20260809 07:49:29.837766 4481 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:29.857422 4481 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:29.858341 4481 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:29.858608 4481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:29.859524 4481 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:29.861189 4481 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:29.861304 4481 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:29.861464 4481 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:29.861577 4481 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:29.897294 4481 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:36997
I20260809 07:49:29.897405 4660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:36997 every 8 connection(s)
I20260809 07:49:29.898801 4481 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:29.907955 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4481
I20260809 07:49:29.908149 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:49:29.916723 4662 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:29.916800 4663 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:29.917105 4662 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:29.917105 4663 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:29.917927 4661 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:29.918111 4661 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:29.918438 4663 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:29.918462 4662 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:29.918685 4661 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:29.920214 3875 ts_manager.cc:194] Registered new tserver with Master: 382ede2e856d4df8b6da244c62cd2c11 (127.26.37.3:36997)
I20260809 07:49:29.920229 3777 ts_manager.cc:194] Registered new tserver with Master: 382ede2e856d4df8b6da244c62cd2c11 (127.26.37.3:36997)
I20260809 07:49:29.920946 3974 ts_manager.cc:194] Registered new tserver with Master: 382ede2e856d4df8b6da244c62cd2c11 (127.26.37.3:36997)
I20260809 07:49:29.921295 3777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:54191
I20260809 07:49:29.933159 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:49:29.940404 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:49:29.940599 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3741
I20260809 07:49:29.941650 3823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:29.973570 4285 heartbeater.cc:499] Master 127.26.37.62:39893 was elected leader, sending a full tablet report...
I20260809 07:49:30.058470 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3741
W20260809 07:49:30.087803 3859 connection.cc:573] server connection from 127.0.0.1:54596 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:49:30.088634 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39893
--webserver_interface=127.26.37.62
--webserver_port=43915
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683 with env {}
W20260809 07:49:30.439666 4671 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:30.440045 4671 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:30.440685 4671 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:49:30.442576 4472 heartbeater.cc:499] Master 127.26.37.62:39893 was elected leader, sending a full tablet report...
W20260809 07:49:30.444211 4472 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:39893 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:39893: connect: Connection refused (error 111)
W20260809 07:49:30.449985 4671 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:30.450058 4671 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:30.450114 4671 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:30.450161 4671 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:30.462235 4671 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=43915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:30.465939 4671 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:30.469340 4671 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:30.478685 4681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:30.478657 4683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:30.483598 4671 server_base.cc:1061] running on GCE node
W20260809 07:49:30.484115 4685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:30.485235 4671 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:30.485503 4671 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:30.485634 4671 hybrid_clock.cc:648] HybridClock initialized: now 1786261770485611 us; error 0 us; skew 500 ppm
I20260809 07:49:30.486129 4671 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:30.491269 4671 webserver.cc:533] Webserver started at http://127.26.37.62:43915/ using document root <none> and password file <none>
I20260809 07:49:30.492334 4671 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:30.492491 4671 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:30.500384 4671 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:49:30.504314 4691 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:30.506244 4671 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:49:30.506438 4671 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "93f500acf72b4a6fb74061968b61acd7"
format_stamp: "Formatted at 2026-08-09 07:49:26 on dist-test-slave-bw88"
I20260809 07:49:30.507346 4671 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:30.566996 4671 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:30.568017 4671 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:30.568339 4671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:30.590188 4671 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39893
I20260809 07:49:30.590217 4763 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39893 every 8 connection(s)
I20260809 07:49:30.591980 4671 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:30.594740 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4671
I20260809 07:49:30.595075 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4112
I20260809 07:49:30.600606 4764 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:30.604903 4764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap starting.
I20260809 07:49:30.605366 4279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:30.623773 4764 log.cc:826] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:30.632913 4764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:30.633564 4764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap complete.
I20260809 07:49:30.640501 4764 raft_consensus.cc:359] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:30.641377 4764 raft_consensus.cc:740] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f500acf72b4a6fb74061968b61acd7, State: Initialized, Role: FOLLOWER
I20260809 07:49:30.642231 4764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:30.643220 4773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:30.643555 4773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:30.644411 4764 sys_catalog.cc:565] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:30.669991 4785 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:30.670208 4785 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: loading cluster ID for follower catalog manager: success
I20260809 07:49:30.674692 4785 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: acquiring CA information for follower catalog manager: success
I20260809 07:49:30.677541 4785 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:30.709134 4671 master_runner.cc:264] No leader master found from master 93f500acf72b4a6fb74061968b61acd7
I20260809 07:49:30.709312 4671 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:49:30.716429 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4112
I20260809 07:49:30.748226 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43731
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--webserver_port=38913
--webserver_interface=127.26.37.1
--time_source=system_unsync with env {}
I20260809 07:49:30.923763 4661 heartbeater.cc:499] Master 127.26.37.62:39893 was elected leader, sending a full tablet report...
I20260809 07:49:30.934742 4717 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "382ede2e856d4df8b6da244c62cd2c11" instance_seqno: 1786261769888879) as {username='slave'} at 127.26.37.3:39453; Asking this server to re-register.
I20260809 07:49:30.935562 4661 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:30.937258 4717 ts_manager.cc:194] Registered new tserver with Master: 382ede2e856d4df8b6da244c62cd2c11 (127.26.37.3:36997)
W20260809 07:49:31.118165 4793 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:31.118515 4793 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:31.118595 4793 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:31.127336 4793 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:31.127539 4793 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:31.137596 4793 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43731
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38913
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:31.139195 4793 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:31.141229 4793 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:31.151957 4804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:31.151983 4803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:31.152554 4806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:31.153966 4793 server_base.cc:1061] running on GCE node
I20260809 07:49:31.154582 4793 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:31.154740 4793 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:31.154809 4793 hybrid_clock.cc:648] HybridClock initialized: now 1786261771154791 us; error 0 us; skew 500 ppm
I20260809 07:49:31.155079 4793 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:31.157279 4793 webserver.cc:533] Webserver started at http://127.26.37.1:38913/ using document root <none> and password file <none>
I20260809 07:49:31.158165 4793 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:31.158294 4793 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:31.164106 4793 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 07:49:31.166828 4811 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:31.168118 4793 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:49:31.168298 4793 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:31.169024 4793 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:31.183387 4793 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:31.184109 4793 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:31.184356 4793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:31.185220 4793 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:31.186769 4793 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:31.186841 4793 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:31.186915 4793 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:31.186956 4793 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:31.222146 4793 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43731
I20260809 07:49:31.222191 4964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43731 every 8 connection(s)
I20260809 07:49:31.223973 4793 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:31.231884 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4793
I20260809 07:49:31.245268 4965 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:31.245733 4965 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:31.246893 4965 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:31.248862 4717 ts_manager.cc:194] Registered new tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
I20260809 07:49:31.251190 4966 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:31.251356 4966 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:31.251840 4966 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:31.253193 3974 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
I20260809 07:49:31.253486 4967 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:31.253659 4967 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:31.254158 4967 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:31.258358 3875 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
I20260809 07:49:31.335397 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4793
I20260809 07:49:31.346117 4959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:31.404604 4983 raft_consensus.cc:493] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 93f500acf72b4a6fb74061968b61acd7)
I20260809 07:49:31.404858 4983 raft_consensus.cc:515] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:31.406402 4983 leader_election.cc:290] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93f500acf72b4a6fb74061968b61acd7 (127.26.37.62:39893), a8574a2832484382b63407f07c2a72ae (127.26.37.61:39555)
I20260809 07:49:31.407742 3889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a8574a2832484382b63407f07c2a72ae" is_pre_election: true
I20260809 07:49:31.413901 4730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "93f500acf72b4a6fb74061968b61acd7" is_pre_election: true
I20260809 07:49:31.414526 4730 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74e2a4f17ac4fd7b32e1d55390fabd4 in term 1.
I20260809 07:49:31.415100 3955 leader_election.cc:304] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93f500acf72b4a6fb74061968b61acd7, d74e2a4f17ac4fd7b32e1d55390fabd4; no voters: a8574a2832484382b63407f07c2a72ae
I20260809 07:49:31.415556 4983 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:49:31.415678 4983 raft_consensus.cc:493] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 93f500acf72b4a6fb74061968b61acd7)
I20260809 07:49:31.415763 4983 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:49:31.418247 4983 raft_consensus.cc:515] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:31.418804 4983 leader_election.cc:290] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [CANDIDATE]: Term 2 election: Requested vote from peers 93f500acf72b4a6fb74061968b61acd7 (127.26.37.62:39893), a8574a2832484382b63407f07c2a72ae (127.26.37.61:39555)
I20260809 07:49:31.419330 4730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "93f500acf72b4a6fb74061968b61acd7"
I20260809 07:49:31.419384 3889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a8574a2832484382b63407f07c2a72ae"
I20260809 07:49:31.419523 4730 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:49:31.425000 4730 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74e2a4f17ac4fd7b32e1d55390fabd4 in term 2.
I20260809 07:49:31.425417 3955 leader_election.cc:304] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93f500acf72b4a6fb74061968b61acd7, d74e2a4f17ac4fd7b32e1d55390fabd4; no voters: a8574a2832484382b63407f07c2a72ae
I20260809 07:49:31.425891 4983 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:49:31.426342 4983 raft_consensus.cc:697] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 2 LEADER]: Becoming Leader. State: Replica: d74e2a4f17ac4fd7b32e1d55390fabd4, State: Running, Role: LEADER
I20260809 07:49:31.426872 4983 consensus_queue.cc:237] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:31.428766 4985 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d74e2a4f17ac4fd7b32e1d55390fabd4. Latest consensus state: current_term: 2 leader_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:31.429059 4985 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:31.429785 4988 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:49:31.431586 4988 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:49:31.432524 4988 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:31.432623 4988 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:49:31.433833 4988 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:49:31.434640 4988 catalog_manager.cc:6103] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Loaded TSK: 0
I20260809 07:49:31.435493 4988 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:49:31.439383 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4793
I20260809 07:49:31.451917 4472 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:31.452129 4472 heartbeater.cc:499] Master 127.26.37.62:39893 was elected leader, sending a full tablet report...
I20260809 07:49:31.453066 4717 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dbbd7c27adec465ba9108196b61f7a3c" instance_seqno: 1786261769405736) as {username='slave'} at 127.26.37.2:37325; Asking this server to re-register.
I20260809 07:49:31.453693 4472 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:31.454802 4717 ts_manager.cc:194] Registered new tserver with Master: dbbd7c27adec465ba9108196b61f7a3c (127.26.37.2:45817)
I20260809 07:49:31.465519 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43731
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--webserver_port=38913
--webserver_interface=127.26.37.1
--time_source=system_unsync
--fault_before_start=1.0 with env {}
I20260809 07:49:31.560652 4992 raft_consensus.cc:493] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 93f500acf72b4a6fb74061968b61acd7)
I20260809 07:49:31.561038 4992 raft_consensus.cc:515] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:31.562470 4992 leader_election.cc:290] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93f500acf72b4a6fb74061968b61acd7 (127.26.37.62:39893), d74e2a4f17ac4fd7b32e1d55390fabd4 (127.26.37.60:37683)
I20260809 07:49:31.563818 3985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8574a2832484382b63407f07c2a72ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" is_pre_election: true
I20260809 07:49:31.569649 4730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8574a2832484382b63407f07c2a72ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "93f500acf72b4a6fb74061968b61acd7" is_pre_election: true
I20260809 07:49:31.570083 4730 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8574a2832484382b63407f07c2a72ae in current term 2: Already voted for candidate d74e2a4f17ac4fd7b32e1d55390fabd4 in this term.
I20260809 07:49:31.570850 3859 leader_election.cc:304] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8574a2832484382b63407f07c2a72ae; no voters: 93f500acf72b4a6fb74061968b61acd7, d74e2a4f17ac4fd7b32e1d55390fabd4
I20260809 07:49:31.571311 4992 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:49:31.573551 4992 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 07:49:31.810539 4991 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:31.810833 4991 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:31.811024 4991 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:49:31.811118 4991 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:31.819157 4991 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:31.819423 4991 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:31.826617 4730 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Refusing update from remote peer d74e2a4f17ac4fd7b32e1d55390fabd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:49:31.827513 4985 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:49:31.831067 4991 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43731
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38913
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:31.831275 4997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d74e2a4f17ac4fd7b32e1d55390fabd4. Latest consensus state: current_term: 2 leader_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:31.831596 4997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:31.833052 4983 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:31.833187 4991 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:31.833700 3889 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 2 FOLLOWER]: Refusing update from remote peer d74e2a4f17ac4fd7b32e1d55390fabd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:49:31.833278 4983 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:31.834414 4983 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:49:31.835479 4997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:31.835641 4991 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:49:31.835778 4997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:31.837174 5007 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:31.837268 4992 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader d74e2a4f17ac4fd7b32e1d55390fabd4. Latest consensus state: current_term: 2 leader_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:31.837483 5007 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:31.837555 4992 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:49:31.845778 5009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:31.845783 5010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:31.846189 5012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:31.847723 4991 server_base.cc:1061] running on GCE node
I20260809 07:49:31.848277 4991 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:31.848445 4991 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:31.848559 4991 hybrid_clock.cc:648] HybridClock initialized: now 1786261771848507 us; error 0 us; skew 500 ppm
I20260809 07:49:31.848865 4991 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:31.851035 4991 webserver.cc:533] Webserver started at http://127.26.37.1:38913/ using document root <none> and password file <none>
I20260809 07:49:31.851763 4991 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:31.851907 4991 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:31.857141 4991 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:49:31.859725 5018 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:31.860951 4991 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:49:31.861166 4991 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:31.861981 4991 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:31.876833 4991 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:31.877689 4991 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:31.877955 4991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:31.878846 4991 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:31.880512 4991 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:31.880643 4991 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:31.880759 4991 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:31.880848 4991 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:49:31.881242 4991 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:49:31.881368 4991 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:49:31.881469 4991 fault_injection.cc:59] injecting fault for kudu (pid 4991): FLAGS_fault_before_start (process will exit)
I20260809 07:49:31.906859 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43731
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--webserver_port=38913
--webserver_interface=127.26.37.1
--time_source=system_unsync with env {}
W20260809 07:49:32.253518 5029 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:32.253961 5029 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:32.254145 5029 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:32.263545 5029 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:32.263705 5029 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:32.273521 5029 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43731
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38913
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:32.275128 5029 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:32.276897 5029 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:32.286756 5038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:32.286907 5037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:32.287571 5040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:32.288393 5029 server_base.cc:1061] running on GCE node
I20260809 07:49:32.288936 5029 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:32.289120 5029 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:32.289212 5029 hybrid_clock.cc:648] HybridClock initialized: now 1786261772289175 us; error 0 us; skew 500 ppm
I20260809 07:49:32.289601 5029 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:32.291918 5029 webserver.cc:533] Webserver started at http://127.26.37.1:38913/ using document root <none> and password file <none>
I20260809 07:49:32.292747 5029 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:32.292893 5029 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:32.298491 5029 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:49:32.301328 5047 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:32.302785 5029 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:32.302983 5029 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:32.303897 5029 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:32.319208 5029 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:32.320022 5029 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:32.320338 5029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:32.321450 5029 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:32.323268 5029 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:32.323350 5029 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:32.323531 5029 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:32.323596 5029 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:32.362865 5029 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43731
I20260809 07:49:32.362906 5200 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43731 every 8 connection(s)
I20260809 07:49:32.364488 5029 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:32.367974 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5029
I20260809 07:49:32.368283 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5029
I20260809 07:49:32.379741 4983 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:32.380115 4983 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:32.400416 5196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:32.444973 4473 heartbeater.cc:499] Master 127.26.37.60:37683 was elected leader, sending a full tablet report...
I20260809 07:49:32.400624 5203 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:32.400701 5202 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:32.488071 5203 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:32.488078 5202 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:32.489383 5202 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:32.489439 5203 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:32.400663 5204 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:32.489657 5204 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:32.490229 5204 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:32.490984 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5029
I20260809 07:49:32.492254 3974 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
I20260809 07:49:32.493412 4717 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
I20260809 07:49:32.494035 3974 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:60611
I20260809 07:49:32.495219 3875 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:43731)
W20260809 07:49:32.516191 3956 connection.cc:573] server connection from 127.26.37.1:60611 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:49:32.516209 3857 connection.cc:573] server connection from 127.26.37.1:36915 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:49:32.516214 4693 connection.cc:573] server connection from 127.26.37.1:57065 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:49:32.516847 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4297
I20260809 07:49:32.524940 4463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:32.610528 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4297
I20260809 07:49:32.635903 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4481
I20260809 07:49:32.644326 4655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:32.729494 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4481
I20260809 07:49:32.753398 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4671
I20260809 07:49:32.759780 4759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:32.868232 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4671
I20260809 07:49:32.896662 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3833
I20260809 07:49:32.897982 3922 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:33.010125 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3833
I20260809 07:49:33.037026 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3929
I20260809 07:49:33.038311 4019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:33.148481 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3929
Shutting down hive metastore.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:49:34: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:49:40.994182 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39893
--webserver_interface=127.26.37.62
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683 with env {}
W20260809 07:49:41.343271 5468 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:41.343644 5468 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:41.343780 5468 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:41.351930 5468 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:41.352056 5468 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:41.352181 5468 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:41.352267 5468 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:41.361984 5468 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:41.363674 5468 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:41.365513 5468 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:41.373703 5476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:41.373771 5478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:41.374768 5480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:41.375403 5468 server_base.cc:1061] running on GCE node
I20260809 07:49:41.376216 5468 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:41.376396 5468 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:41.376506 5468 hybrid_clock.cc:648] HybridClock initialized: now 1786261781376461 us; error 0 us; skew 500 ppm
I20260809 07:49:41.376827 5468 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:41.379330 5468 webserver.cc:533] Webserver started at http://127.26.37.62:45915/ using document root <none> and password file <none>
I20260809 07:49:41.380026 5468 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:41.380154 5468 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:41.385321 5468 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:49:41.387952 5488 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:41.389138 5468 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:49:41.389317 5468 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "93f500acf72b4a6fb74061968b61acd7"
format_stamp: "Formatted at 2026-08-09 07:49:26 on dist-test-slave-bw88"
I20260809 07:49:41.390056 5468 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:41.406734 5468 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:41.407541 5468 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:41.407823 5468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:41.427687 5468 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39893
I20260809 07:49:41.427768 5558 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39893 every 8 connection(s)
I20260809 07:49:41.429390 5468 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:41.436327 5559 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:41.437480 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5468
I20260809 07:49:41.437660 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:49:41.438840 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39555
--webserver_interface=127.26.37.61
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683 with env {}
I20260809 07:49:41.440865 5559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap starting.
I20260809 07:49:41.459010 5559 log.cc:826] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:41.467893 5559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:41.468427 5559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap complete.
I20260809 07:49:41.475062 5559 raft_consensus.cc:359] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:41.475736 5559 raft_consensus.cc:740] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f500acf72b4a6fb74061968b61acd7, State: Initialized, Role: FOLLOWER
I20260809 07:49:41.476406 5559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:41.477274 5564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:41.477607 5564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:41.478293 5559 sys_catalog.cc:565] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:41.488430 5579 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:41.488597 5579 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: loading cluster ID for follower catalog manager: success
I20260809 07:49:41.493423 5579 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: acquiring CA information for follower catalog manager: success
I20260809 07:49:41.497248 5579 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:41.508075 5468 master_runner.cc:187] Error getting master registration for 127.26.37.61:39555: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39555: connect: Connection refused (error 111),
I20260809 07:49:41.511884 5468 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:49:41.807554 5562 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:41.807895 5562 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:41.808027 5562 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:41.816035 5562 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:41.816162 5562 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:41.816277 5562 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:41.816358 5562 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:41.826012 5562 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:41.827775 5562 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:41.829741 5562 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:41.837608 5588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:41.838394 5587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:41.838711 5593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:41.840080 5562 server_base.cc:1061] running on GCE node
I20260809 07:49:41.840925 5562 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:41.841104 5562 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:41.841216 5562 hybrid_clock.cc:648] HybridClock initialized: now 1786261781841178 us; error 0 us; skew 500 ppm
I20260809 07:49:41.841630 5562 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:41.844800 5562 webserver.cc:533] Webserver started at http://127.26.37.61:37255/ using document root <none> and password file <none>
I20260809 07:49:41.845765 5562 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:41.845911 5562 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:41.852932 5562 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:49:41.856302 5600 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:41.857925 5562 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:49:41.858106 5562 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "a8574a2832484382b63407f07c2a72ae"
format_stamp: "Formatted at 2026-08-09 07:49:27 on dist-test-slave-bw88"
I20260809 07:49:41.859138 5562 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:41.893250 5562 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:41.894351 5562 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:41.894680 5562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:41.915553 5562 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:39555
I20260809 07:49:41.915575 5665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:39555 every 8 connection(s)
I20260809 07:49:41.917294 5562 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:49:41.924067 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5562
I20260809 07:49:41.924244 5666 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:41.924291 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:49:41.925478 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37683
--webserver_interface=127.26.37.60
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683 with env {}
I20260809 07:49:41.929226 5666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: Bootstrap starting.
I20260809 07:49:41.948374 5666 log.cc:826] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:41.957502 5666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:41.957994 5666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: Bootstrap complete.
I20260809 07:49:41.964676 5666 raft_consensus.cc:359] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:41.965344 5666 raft_consensus.cc:740] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8574a2832484382b63407f07c2a72ae, State: Initialized, Role: FOLLOWER
I20260809 07:49:41.966130 5666 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:41.967000 5670 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:41.967321 5670 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:41.968015 5666 sys_catalog.cc:565] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:41.978349 5685 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:41.978525 5685 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: loading cluster ID for follower catalog manager: success
I20260809 07:49:41.983302 5685 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: acquiring CA information for follower catalog manager: success
I20260809 07:49:41.985872 5685 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:42.011314 5562 master_runner.cc:187] Error getting master registration for 127.26.37.60:37683: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37683: connect: Connection refused (error 111),
I20260809 07:49:42.017238 5562 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:49:42.282685 5668 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:42.283044 5668 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:42.283172 5668 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:42.291523 5668 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:42.291636 5668 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:42.291746 5668 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:42.291828 5668 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:42.301880 5668 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:42.303562 5668 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:42.305505 5668 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:42.313933 5699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:42.314481 5698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:42.315091 5702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:42.316345 5668 server_base.cc:1061] running on GCE node
I20260809 07:49:42.317194 5668 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:42.317405 5668 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:42.317492 5668 hybrid_clock.cc:648] HybridClock initialized: now 1786261782317472 us; error 0 us; skew 500 ppm
I20260809 07:49:42.317844 5668 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:42.320889 5668 webserver.cc:533] Webserver started at http://127.26.37.60:36403/ using document root <none> and password file <none>
I20260809 07:49:42.321821 5668 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:42.321983 5668 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:42.328884 5668 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 07:49:42.332484 5707 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:42.334193 5668 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:42.334367 5668 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4"
format_stamp: "Formatted at 2026-08-09 07:49:27 on dist-test-slave-bw88"
I20260809 07:49:42.335351 5668 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:42.363965 5668 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:42.365296 5668 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:42.365732 5668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:42.388514 5668 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:37683
I20260809 07:49:42.388550 5784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:37683 every 8 connection(s)
I20260809 07:49:42.390362 5668 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:49:42.390877 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5668
I20260809 07:49:42.391062 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:49:42.399893 5785 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:42.404299 5785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Bootstrap starting.
I20260809 07:49:42.423738 5785 log.cc:826] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:42.432832 5785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:42.433439 5785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: Bootstrap complete.
I20260809 07:49:42.440146 5785 raft_consensus.cc:359] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:42.441022 5785 raft_consensus.cc:740] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74e2a4f17ac4fd7b32e1d55390fabd4, State: Initialized, Role: FOLLOWER
I20260809 07:49:42.441756 5785 consensus_queue.cc:260] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:42.442728 5790 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:42.443010 5790 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:42.443770 5785 sys_catalog.cc:565] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:42.454989 5802 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:42.455112 5802 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: loading cluster ID for follower catalog manager: success
I20260809 07:49:42.458595 5802 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: acquiring CA information for follower catalog manager: success
I20260809 07:49:42.461190 5802 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:42.482318 5668 master_runner.cc:264] No leader master found from master d74e2a4f17ac4fd7b32e1d55390fabd4
I20260809 07:49:42.482517 5668 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:49:43.031402 5809 raft_consensus.cc:493] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:49:43.031893 5809 raft_consensus.cc:515] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:43.039505 5809 leader_election.cc:290] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a8574a2832484382b63407f07c2a72ae (127.26.37.61:39555), d74e2a4f17ac4fd7b32e1d55390fabd4 (127.26.37.60:37683)
I20260809 07:49:43.045652 5750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93f500acf72b4a6fb74061968b61acd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" is_pre_election: true
I20260809 07:49:43.046271 5750 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f500acf72b4a6fb74061968b61acd7 in term 2.
I20260809 07:49:43.046919 5491 leader_election.cc:304] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93f500acf72b4a6fb74061968b61acd7, d74e2a4f17ac4fd7b32e1d55390fabd4; no voters:
I20260809 07:49:43.047322 5809 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:49:43.047528 5809 raft_consensus.cc:493] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:49:43.047641 5809 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:49:43.047456 5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93f500acf72b4a6fb74061968b61acd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a8574a2832484382b63407f07c2a72ae" is_pre_election: true
I20260809 07:49:43.048017 5635 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f500acf72b4a6fb74061968b61acd7 in term 2.
I20260809 07:49:43.051542 5809 raft_consensus.cc:515] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:43.052124 5809 leader_election.cc:290] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [CANDIDATE]: Term 3 election: Requested vote from peers a8574a2832484382b63407f07c2a72ae (127.26.37.61:39555), d74e2a4f17ac4fd7b32e1d55390fabd4 (127.26.37.60:37683)
I20260809 07:49:43.052650 5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93f500acf72b4a6fb74061968b61acd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a8574a2832484382b63407f07c2a72ae"
I20260809 07:49:43.052841 5635 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:49:43.052803 5750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93f500acf72b4a6fb74061968b61acd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4"
I20260809 07:49:43.053052 5750 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:49:43.056947 5635 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f500acf72b4a6fb74061968b61acd7 in term 3.
I20260809 07:49:43.057437 5491 leader_election.cc:304] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93f500acf72b4a6fb74061968b61acd7, a8574a2832484382b63407f07c2a72ae; no voters:
I20260809 07:49:43.057449 5750 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f500acf72b4a6fb74061968b61acd7 in term 3.
I20260809 07:49:43.057720 5809 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:49:43.058207 5809 raft_consensus.cc:697] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 3 LEADER]: Becoming Leader. State: Replica: 93f500acf72b4a6fb74061968b61acd7, State: Running, Role: LEADER
I20260809 07:49:43.058764 5809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:43.061493 5817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93f500acf72b4a6fb74061968b61acd7. Latest consensus state: current_term: 3 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:43.061744 5817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:43.062403 5819 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:49:43.064213 5819 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:49:43.064937 5819 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:43.065017 5819 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:49:43.066041 5819 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:49:43.066771 5819 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Loaded TSK: 0
I20260809 07:49:43.067541 5819 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:49:43.097818 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--time_source=system_unsync with env {}
W20260809 07:49:43.431906 5821 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:43.432199 5821 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:43.432340 5821 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:43.440598 5821 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:43.440774 5821 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:43.450198 5821 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:43.451699 5821 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:43.453517 5821 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:43.462375 5827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:43.462404 5829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:43.462721 5831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:43.464192 5821 server_base.cc:1061] running on GCE node
I20260809 07:49:43.464708 5821 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:43.464848 5821 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:43.464908 5821 hybrid_clock.cc:648] HybridClock initialized: now 1786261783464892 us; error 0 us; skew 500 ppm
I20260809 07:49:43.465135 5821 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:43.467372 5821 webserver.cc:533] Webserver started at http://127.26.37.1:41063/ using document root <none> and password file <none>
I20260809 07:49:43.468117 5821 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:43.468214 5821 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:43.473479 5821 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:49:43.475945 5837 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:43.477192 5821 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:49:43.477344 5821 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:43.478122 5821 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:43.507472 5821 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:43.508299 5821 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:43.508598 5821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:43.509586 5821 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:43.511288 5821 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:43.511359 5821 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:43.511528 5821 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:43.511588 5821 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:43.547438 5821 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:38085
I20260809 07:49:43.547473 5991 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:38085 every 8 connection(s)
I20260809 07:49:43.549079 5821 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:43.549472 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5821
I20260809 07:49:43.549687 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:49:43.557013 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--time_source=system_unsync with env {}
I20260809 07:49:43.584373 5993 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:43.584517 5992 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:43.584838 5993 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:43.584836 5992 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:43.586138 5992 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:43.586165 5993 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:43.588737 5508 ts_manager.cc:194] Registered new tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:43.588784 5738 ts_manager.cc:194] Registered new tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:43.591037 5994 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:43.591195 5994 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:43.591249 5508 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:59759
I20260809 07:49:43.591758 5994 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:43.593514 5622 ts_manager.cc:194] Registered new tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:43.664943 5750 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 3 FOLLOWER]: Refusing update from remote peer 93f500acf72b4a6fb74061968b61acd7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:49:43.665518 5635 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 3 FOLLOWER]: Refusing update from remote peer 93f500acf72b4a6fb74061968b61acd7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:49:43.665918 6005 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:49:43.666726 6006 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:49:43.669687 6007 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93f500acf72b4a6fb74061968b61acd7. Latest consensus state: current_term: 3 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:43.670064 6007 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:43.671118 6010 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93f500acf72b4a6fb74061968b61acd7. Latest consensus state: current_term: 3 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:43.671484 6010 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:43.672662 6006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:43.672912 6006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:43.672935 6007 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:43.673120 6007 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:43.674113 6005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:43.674387 6005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:43.675103 6010 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "93f500acf72b4a6fb74061968b61acd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:43.675279 6010 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:49:43.925061 6004 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:43.925426 6004 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:43.925544 6004 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:43.936022 6004 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:43.936209 6004 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:49:43.945410 6004 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:43.946955 6004 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:43.948691 6004 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:43.958388 6024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:43.958480 6025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:43.958647 6027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:43.959184 6004 server_base.cc:1061] running on GCE node
I20260809 07:49:43.959771 6004 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:43.959903 6004 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:43.959964 6004 hybrid_clock.cc:648] HybridClock initialized: now 1786261783959948 us; error 0 us; skew 500 ppm
I20260809 07:49:43.960194 6004 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:43.962298 6004 webserver.cc:533] Webserver started at http://127.26.37.2:37525/ using document root <none> and password file <none>
I20260809 07:49:43.962980 6004 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:43.963095 6004 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:43.968214 6004 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 07:49:43.971005 6035 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:43.972394 6004 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 07:49:43.972570 6004 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "dbbd7c27adec465ba9108196b61f7a3c"
format_stamp: "Formatted at 2026-08-09 07:49:29 on dist-test-slave-bw88"
I20260809 07:49:43.973268 6004 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:43.989711 6004 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:43.990448 6004 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:43.990739 6004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:43.991559 6004 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:43.993096 6004 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:43.993208 6004 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:43.993422 6004 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:43.993510 6004 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 07:49:44.028239 6004 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:36083
I20260809 07:49:44.028306 6180 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:36083 every 8 connection(s)
I20260809 07:49:44.029908 6004 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:49:44.042904 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6004
I20260809 07:49:44.043392 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:49:44.048280 6183 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:44.048686 6183 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:44.048900 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--time_source=system_unsync with env {}
I20260809 07:49:44.049400 6181 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:44.049561 6181 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:44.049557 6182 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:44.049762 6182 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:44.049841 6183 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:44.050251 6181 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:44.050303 6182 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:44.051723 5738 ts_manager.cc:194] Registered new tserver with Master: dbbd7c27adec465ba9108196b61f7a3c (127.26.37.2:36083)
I20260809 07:49:44.051766 5508 ts_manager.cc:194] Registered new tserver with Master: dbbd7c27adec465ba9108196b61f7a3c (127.26.37.2:36083)
I20260809 07:49:44.051949 5622 ts_manager.cc:194] Registered new tserver with Master: dbbd7c27adec465ba9108196b61f7a3c (127.26.37.2:36083)
I20260809 07:49:44.052855 5508 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:35673
W20260809 07:49:44.394462 6189 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:44.394799 6189 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:44.394922 6189 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:44.403039 6189 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:44.403239 6189 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:49:44.412699 6189 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:44.414394 6189 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:44.416281 6189 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:44.426569 6196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:44.426407 6195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:44.426759 6199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:44.429749 6189 server_base.cc:1061] running on GCE node
I20260809 07:49:44.430411 6189 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:44.430598 6189 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:44.430693 6189 hybrid_clock.cc:648] HybridClock initialized: now 1786261784430661 us; error 0 us; skew 500 ppm
I20260809 07:49:44.431005 6189 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:44.433408 6189 webserver.cc:533] Webserver started at http://127.26.37.3:40729/ using document root <none> and password file <none>
I20260809 07:49:44.434208 6189 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:44.434350 6189 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:44.439798 6189 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 07:49:44.442672 6210 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:44.444145 6189 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:49:44.444315 6189 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "382ede2e856d4df8b6da244c62cd2c11"
format_stamp: "Formatted at 2026-08-09 07:49:29 on dist-test-slave-bw88"
I20260809 07:49:44.445103 6189 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:44.481432 6189 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:44.482288 6189 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:44.482578 6189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:44.483600 6189 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:44.485224 6189 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:44.485330 6189 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:44.485482 6189 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:44.485551 6189 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:44.520639 6189 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:42745
I20260809 07:49:44.520727 6351 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:42745 every 8 connection(s)
I20260809 07:49:44.522296 6189 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:44.532255 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6189
I20260809 07:49:44.532465 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:49:44.540133 6356 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:44.540264 6354 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:44.540292 6355 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:44.540663 6355 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:44.540673 6354 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:44.541585 6356 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:44.541800 6354 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:44.541836 6355 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:44.542232 6356 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:44.543638 5508 ts_manager.cc:194] Registered new tserver with Master: 382ede2e856d4df8b6da244c62cd2c11 (127.26.37.3:42745)
I20260809 07:49:44.543666 5738 ts_manager.cc:194] Registered new tserver with Master: 382ede2e856d4df8b6da244c62cd2c11 (127.26.37.3:42745)
I20260809 07:49:44.543715 5622 ts_manager.cc:194] Registered new tserver with Master: 382ede2e856d4df8b6da244c62cd2c11 (127.26.37.3:42745)
I20260809 07:49:44.544703 5508 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:55583
I20260809 07:49:44.557451 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:49:44.564450 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:49:44.564649 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5468
I20260809 07:49:44.565516 5551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:44.594389 5992 heartbeater.cc:499] Master 127.26.37.62:39893 was elected leader, sending a full tablet report...
I20260809 07:49:44.673630 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5468
W20260809 07:49:44.702677 5710 connection.cc:573] server connection from 127.0.0.1:54678 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:49:44.703446 5992 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:39893 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.26.37.62:39893 (error 108)
I20260809 07:49:44.703943 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39893
--webserver_interface=127.26.37.62
--webserver_port=45915
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683 with env {}
W20260809 07:49:45.048755 6363 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:45.049093 6363 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:45.049228 6363 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:49:45.055467 6181 heartbeater.cc:499] Master 127.26.37.62:39893 was elected leader, sending a full tablet report...
W20260809 07:49:45.056923 6181 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:39893 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:39893: connect: Connection refused (error 111)
W20260809 07:49:45.058048 6363 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:45.058156 6363 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:45.058266 6363 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:45.058341 6363 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:45.068289 6363 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=45915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:45.070075 6363 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:45.071980 6363 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:45.080119 6371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:45.080650 6374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:45.080120 6372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:45.081451 6363 server_base.cc:1061] running on GCE node
I20260809 07:49:45.082142 6363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:45.082324 6363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:45.082414 6363 hybrid_clock.cc:648] HybridClock initialized: now 1786261785082379 us; error 0 us; skew 500 ppm
I20260809 07:49:45.082785 6363 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:45.088316 6363 webserver.cc:533] Webserver started at http://127.26.37.62:45915/ using document root <none> and password file <none>
I20260809 07:49:45.089298 6363 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:45.089509 6363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:45.095151 6363 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:49:45.097855 6382 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:45.099184 6363 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:45.099316 6363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "93f500acf72b4a6fb74061968b61acd7"
format_stamp: "Formatted at 2026-08-09 07:49:26 on dist-test-slave-bw88"
I20260809 07:49:45.100073 6363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:45.112661 6363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:45.113560 6363 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:45.113803 6363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:45.134053 6363 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39893
I20260809 07:49:45.134094 6442 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39893 every 8 connection(s)
I20260809 07:49:45.135720 6363 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:45.142953 6444 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:45.144994 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6363
I20260809 07:49:45.145303 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5821
I20260809 07:49:45.147269 6444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap starting.
I20260809 07:49:45.155035 5986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:45.167936 6444 log.cc:826] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:45.177691 6444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:45.178256 6444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: Bootstrap complete.
I20260809 07:49:45.185312 6444 raft_consensus.cc:359] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:45.186084 6444 raft_consensus.cc:740] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f500acf72b4a6fb74061968b61acd7, State: Initialized, Role: FOLLOWER
I20260809 07:49:45.186822 6444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:45.187749 6451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:45.188114 6451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:45.188915 6444 sys_catalog.cc:565] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:45.199712 6465 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:45.199908 6465 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: loading cluster ID for follower catalog manager: success
I20260809 07:49:45.204960 6465 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: acquiring CA information for follower catalog manager: success
I20260809 07:49:45.210614 6465 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:49:45.241750 6363 master_runner.cc:264] No leader master found from master 93f500acf72b4a6fb74061968b61acd7
I20260809 07:49:45.241921 6363 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:49:45.263259 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5821
I20260809 07:49:45.302454 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:38085
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--webserver_port=41063
--webserver_interface=127.26.37.1
--time_source=system_unsync with env {}
I20260809 07:49:45.547151 6354 heartbeater.cc:499] Master 127.26.37.62:39893 was elected leader, sending a full tablet report...
I20260809 07:49:45.570456 6405 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "382ede2e856d4df8b6da244c62cd2c11" instance_seqno: 1786261784512315) as {username='slave'} at 127.26.37.3:55739; Asking this server to re-register.
I20260809 07:49:45.571831 6354 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:45.573987 6405 ts_manager.cc:194] Registered new tserver with Master: 382ede2e856d4df8b6da244c62cd2c11 (127.26.37.3:42745)
I20260809 07:49:45.631794 6475 raft_consensus.cc:493] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 93f500acf72b4a6fb74061968b61acd7)
I20260809 07:49:45.632114 6475 raft_consensus.cc:515] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:45.634567 6475 leader_election.cc:290] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 93f500acf72b4a6fb74061968b61acd7 (127.26.37.62:39893), d74e2a4f17ac4fd7b32e1d55390fabd4 (127.26.37.60:37683)
I20260809 07:49:45.642572 6417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8574a2832484382b63407f07c2a72ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "93f500acf72b4a6fb74061968b61acd7" is_pre_election: true
I20260809 07:49:45.643149 6417 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8574a2832484382b63407f07c2a72ae in term 3.
I20260809 07:49:45.643388 5750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8574a2832484382b63407f07c2a72ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" is_pre_election: true
I20260809 07:49:45.643894 5603 leader_election.cc:304] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93f500acf72b4a6fb74061968b61acd7, a8574a2832484382b63407f07c2a72ae; no voters:
I20260809 07:49:45.644291 6475 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:49:45.644438 6475 raft_consensus.cc:493] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 3 FOLLOWER]: Starting leader election (detected failure of leader 93f500acf72b4a6fb74061968b61acd7)
I20260809 07:49:45.644585 6475 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:49:45.648034 6475 raft_consensus.cc:515] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:45.648804 6475 leader_election.cc:290] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [CANDIDATE]: Term 4 election: Requested vote from peers 93f500acf72b4a6fb74061968b61acd7 (127.26.37.62:39893), d74e2a4f17ac4fd7b32e1d55390fabd4 (127.26.37.60:37683)
I20260809 07:49:45.649199 6417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8574a2832484382b63407f07c2a72ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "93f500acf72b4a6fb74061968b61acd7"
I20260809 07:49:45.649406 6417 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:49:45.649379 5750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8574a2832484382b63407f07c2a72ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4"
I20260809 07:49:45.655184 6417 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8574a2832484382b63407f07c2a72ae in term 4.
I20260809 07:49:45.655644 5603 leader_election.cc:304] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93f500acf72b4a6fb74061968b61acd7, a8574a2832484382b63407f07c2a72ae; no voters: d74e2a4f17ac4fd7b32e1d55390fabd4
I20260809 07:49:45.655979 6475 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:49:45.656428 6475 raft_consensus.cc:697] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [term 4 LEADER]: Becoming Leader. State: Replica: a8574a2832484382b63407f07c2a72ae, State: Running, Role: LEADER
I20260809 07:49:45.656948 6475 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:45.659049 6478 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8574a2832484382b63407f07c2a72ae. Latest consensus state: current_term: 4 leader_uuid: "a8574a2832484382b63407f07c2a72ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:45.659374 6478 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:45.660089 6480 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:49:45.662101 6480 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:49:45.662890 6480 catalog_manager.cc:1295] Loaded cluster ID: 7103933aff4042978042a3b57bd15ce2
I20260809 07:49:45.662967 6480 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:49:45.664109 6480 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:49:45.664876 6480 catalog_manager.cc:6103] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae: Loaded TSK: 0
I20260809 07:49:45.665761 6480 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260809 07:49:45.753031 6472 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:45.753394 6472 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:45.753541 6472 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:45.761650 6472 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:45.761861 6472 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:45.771669 6472 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:38085
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41063
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:45.773304 6472 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:45.775207 6472 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:45.784806 6486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:45.784868 6491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:45.785527 6495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:45.786623 6472 server_base.cc:1061] running on GCE node
I20260809 07:49:45.787178 6472 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:45.787356 6472 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:45.787444 6472 hybrid_clock.cc:648] HybridClock initialized: now 1786261785787424 us; error 0 us; skew 500 ppm
I20260809 07:49:45.787714 6472 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:45.789808 6472 webserver.cc:533] Webserver started at http://127.26.37.1:41063/ using document root <none> and password file <none>
I20260809 07:49:45.790671 6472 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:45.790803 6472 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:45.796020 6472 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 07:49:45.798522 6500 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:45.799723 6472 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:45.799888 6472 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:45.800602 6472 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:45.818638 6472 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:45.819394 6472 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:45.819687 6472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:45.820659 6472 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:45.822410 6472 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:45.822513 6472 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:45.822661 6472 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:45.822732 6472 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:45.857687 6472 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:38085
I20260809 07:49:45.857728 6641 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:38085 every 8 connection(s)
I20260809 07:49:45.859218 6472 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:45.865619 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6472
I20260809 07:49:45.881531 6644 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:45.882009 6644 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:45.883118 6643 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:45.883214 6644 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:45.883276 6643 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:45.883893 6643 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:45.884007 6645 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:45.884132 6645 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:45.884661 6645 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:45.885282 6405 ts_manager.cc:194] Registered new tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:45.885842 5622 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:45.886242 5738 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:45.887724 5622 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:47589
I20260809 07:49:45.982071 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6472
I20260809 07:49:45.991914 6637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:46.063781 6181 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:46.063997 6181 heartbeater.cc:499] Master 127.26.37.62:39893 was elected leader, sending a full tablet report...
I20260809 07:49:46.064845 6405 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dbbd7c27adec465ba9108196b61f7a3c" instance_seqno: 1786261784019954) as {username='slave'} at 127.26.37.2:55799; Asking this server to re-register.
I20260809 07:49:46.065444 6181 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:46.066536 6405 ts_manager.cc:194] Registered new tserver with Master: dbbd7c27adec465ba9108196b61f7a3c (127.26.37.2:36083)
I20260809 07:49:46.082454 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6472
I20260809 07:49:46.108510 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:38085
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--webserver_port=41063
--webserver_interface=127.26.37.1
--time_source=system_unsync
--fault_before_start=1.0 with env {}
I20260809 07:49:46.174724 6658 raft_consensus.cc:493] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 93f500acf72b4a6fb74061968b61acd7)
I20260809 07:49:46.175016 6658 raft_consensus.cc:515] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } }
I20260809 07:49:46.176662 6658 leader_election.cc:290] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 93f500acf72b4a6fb74061968b61acd7 (127.26.37.62:39893), a8574a2832484382b63407f07c2a72ae (127.26.37.61:39555)
I20260809 07:49:46.178128 5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a8574a2832484382b63407f07c2a72ae" is_pre_election: true
I20260809 07:49:46.184132 6417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "93f500acf72b4a6fb74061968b61acd7" is_pre_election: true
I20260809 07:49:46.184588 6417 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74e2a4f17ac4fd7b32e1d55390fabd4 in current term 4: Already voted for candidate a8574a2832484382b63407f07c2a72ae in this term.
I20260809 07:49:46.185372 5715 leader_election.cc:304] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74e2a4f17ac4fd7b32e1d55390fabd4; no voters: 93f500acf72b4a6fb74061968b61acd7, a8574a2832484382b63407f07c2a72ae
I20260809 07:49:46.185766 6658 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:49:46.187971 6658 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:49:46.232650 5750 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [term 4 FOLLOWER]: Refusing update from remote peer a8574a2832484382b63407f07c2a72ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 07:49:46.233457 6660 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:49:46.235347 6658 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8574a2832484382b63407f07c2a72ae. Latest consensus state: current_term: 4 leader_uuid: "a8574a2832484382b63407f07c2a72ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:46.235759 6658 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:46.239279 6417 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [term 4 FOLLOWER]: Refusing update from remote peer a8574a2832484382b63407f07c2a72ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 07:49:46.245702 6663 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "a8574a2832484382b63407f07c2a72ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:46.246991 6663 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:46.248664 6658 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "a8574a2832484382b63407f07c2a72ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:46.247640 6668 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 07:49:46.249039 6658 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74e2a4f17ac4fd7b32e1d55390fabd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:46.253301 6674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "a8574a2832484382b63407f07c2a72ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:46.253844 6674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:49:46.254186 6670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8574a2832484382b63407f07c2a72ae. Latest consensus state: current_term: 4 leader_uuid: "a8574a2832484382b63407f07c2a72ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:46.254458 6670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93f500acf72b4a6fb74061968b61acd7 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:49:46.462623 6657 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:46.462905 6657 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:46.463037 6657 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:49:46.463133 6657 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:46.471349 6657 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:46.471524 6657 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:46.480957 6657 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:38085
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41063
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:46.482750 6657 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:46.484524 6657 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:46.494540 6680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:46.494634 6679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:46.494637 6683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:46.496064 6657 server_base.cc:1061] running on GCE node
I20260809 07:49:46.496655 6657 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:46.496779 6657 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:46.496856 6657 hybrid_clock.cc:648] HybridClock initialized: now 1786261786496825 us; error 0 us; skew 500 ppm
I20260809 07:49:46.497099 6657 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:46.499488 6657 webserver.cc:533] Webserver started at http://127.26.37.1:41063/ using document root <none> and password file <none>
I20260809 07:49:46.500243 6657 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:46.500355 6657 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:46.506011 6657 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:49:46.508661 6689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:46.510170 6657 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:46.510346 6657 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:46.511138 6657 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:46.529554 6657 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:46.530511 6657 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:46.530791 6657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:46.531770 6657 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:46.533550 6657 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:46.533690 6657 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:46.533803 6657 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:46.533893 6657 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:49:46.534255 6657 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:49:46.534365 6657 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:49:46.534441 6657 fault_injection.cc:59] injecting fault for kudu (pid 6657): FLAGS_fault_before_start (process will exit)
I20260809 07:49:46.561486 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:38085
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--webserver_port=41063
--webserver_interface=127.26.37.1
--time_source=system_unsync with env {}
I20260809 07:49:46.648953 6667 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "a8574a2832484382b63407f07c2a72ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f500acf72b4a6fb74061968b61acd7" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39893 } } peers { permanent_uuid: "a8574a2832484382b63407f07c2a72ae" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39555 } } peers { permanent_uuid: "d74e2a4f17ac4fd7b32e1d55390fabd4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37683 } } }
I20260809 07:49:46.649278 6667 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8574a2832484382b63407f07c2a72ae [sys.catalog]: This master's current role is: LEADER
W20260809 07:49:46.906891 6704 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:46.907212 6704 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:46.907352 6704 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:46.915839 6704 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:46.916028 6704 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:49:46.926275 6704 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:38085
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41063
--tserver_master_addrs=127.26.37.62:39893,127.26.37.61:39555,127.26.37.60:37683
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:49:46.927948 6704 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:46.929880 6704 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:49:46.939623 6712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:46.939817 6710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:46.940222 6714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:46.941166 6704 server_base.cc:1061] running on GCE node
I20260809 07:49:46.941754 6704 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:49:46.941942 6704 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:49:46.942039 6704 hybrid_clock.cc:648] HybridClock initialized: now 1786261786942003 us; error 0 us; skew 500 ppm
I20260809 07:49:46.942389 6704 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:49:46.944603 6704 webserver.cc:533] Webserver started at http://127.26.37.1:41063/ using document root <none> and password file <none>
I20260809 07:49:46.945490 6704 fs_manager.cc:362] Metadata directory not provided
I20260809 07:49:46.945616 6704 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:46.951066 6704 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:49:46.953977 6720 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:46.955399 6704 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:46.955559 6704 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f0ab8b2450604e54b121358acfa5ef1d"
format_stamp: "Formatted at 2026-08-09 07:49:28 on dist-test-slave-bw88"
I20260809 07:49:46.956347 6704 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:46.974452 6704 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:49:46.975292 6704 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:46.975575 6704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:46.976485 6704 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:49:46.978206 6704 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:46.978278 6704 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:46.978446 6704 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:49:46.978518 6704 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:47.013235 6704 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:38085
I20260809 07:49:47.013319 6861 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:38085 every 8 connection(s)
I20260809 07:49:47.014798 6704 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:47.015291 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6704
I20260809 07:49:47.015755 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6704
I20260809 07:49:47.042176 6854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:47.057029 6183 heartbeater.cc:499] Master 127.26.37.61:39555 was elected leader, sending a full tablet report...
I20260809 07:49:47.132517 6864 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37683
I20260809 07:49:47.133208 6864 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:47.134652 6864 heartbeater.cc:507] Master 127.26.37.60:37683 requested a full tablet report, sending...
I20260809 07:49:47.136377 6863 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39893
I20260809 07:49:47.136519 6863 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:47.136880 5738 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:47.137044 6863 heartbeater.cc:507] Master 127.26.37.62:39893 requested a full tablet report, sending...
I20260809 07:49:47.139241 6405 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:47.139539 6865 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39555
I20260809 07:49:47.139690 6865 heartbeater.cc:461] Registering TS with master...
I20260809 07:49:47.140159 6865 heartbeater.cc:507] Master 127.26.37.61:39555 requested a full tablet report, sending...
I20260809 07:49:47.141040 5622 ts_manager.cc:194] Re-registered known tserver with Master: f0ab8b2450604e54b121358acfa5ef1d (127.26.37.1:38085)
I20260809 07:49:47.141904 5622 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:53551
I20260809 07:49:47.145740 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6704
I20260809 07:49:47.170553 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6004
I20260809 07:49:47.179059 6175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:47.261982 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6004
I20260809 07:49:47.287848 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6189
I20260809 07:49:47.296949 6346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:47.379876 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6189
I20260809 07:49:47.400861 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6363
I20260809 07:49:47.407253 6438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:47.509745 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6363
W20260809 07:49:47.539189 5715 connection.cc:573] client connection to 127.26.37.62:39893 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:49:47.539558 5668 master_runner.cc:217] Error getting master consensus for 127.26.37.62:39893: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:49:47.539744 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5562
I20260809 07:49:47.539855 5668 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:49:47.540966 5660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:47.652779 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5562
I20260809 07:49:47.682391 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5668
I20260809 07:49:47.683633 5777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:49:47.788539 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5668
Shutting down hive metastore.
[ OK ] ExternalMiniClusterTest.TestBasicOperation/3 (37784 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/4
2026-08-09T07:49:48Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:48Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:49:55: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:50:04.304983 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34301
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093 with env {}
W20260809 07:50:04.642506 7256 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:04.642823 7256 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:04.642910 7256 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:04.650780 7256 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:04.650868 7256 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:04.650921 7256 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:04.650977 7256 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:04.660712 7256 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:04.662421 7256 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:04.664201 7256 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:04.672535 7262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:04.673118 7261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:04.673642 7265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:04.674116 7256 server_base.cc:1061] running on GCE node
I20260809 07:50:04.674993 7256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:04.676481 7256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:04.677713 7256 hybrid_clock.cc:648] HybridClock initialized: now 1786261804677662 us; error 61 us; skew 500 ppm
I20260809 07:50:04.678072 7256 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:04.680280 7256 webserver.cc:533] Webserver started at http://127.26.37.62:43769/ using document root <none> and password file <none>
I20260809 07:50:04.681041 7256 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:04.681211 7256 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:04.681665 7256 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:04.685802 7256 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "cd6de27bda874235b6a67da26180ff6a"
format_stamp: "Formatted at 2026-08-09 07:50:04 on dist-test-slave-bw88"
I20260809 07:50:04.686625 7256 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "cd6de27bda874235b6a67da26180ff6a"
format_stamp: "Formatted at 2026-08-09 07:50:04 on dist-test-slave-bw88"
I20260809 07:50:04.692144 7256 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 07:50:04.695648 7271 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:04.696978 7256 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:50:04.697297 7256 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cd6de27bda874235b6a67da26180ff6a"
format_stamp: "Formatted at 2026-08-09 07:50:04 on dist-test-slave-bw88"
I20260809 07:50:04.697590 7256 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:04.715333 7256 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:04.716218 7256 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:04.716514 7256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:04.736763 7256 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34301
I20260809 07:50:04.736783 7335 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34301 every 8 connection(s)
I20260809 07:50:04.738294 7256 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:04.738652 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7256
I20260809 07:50:04.738864 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:50:04.740222 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38079
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093 with env {}
I20260809 07:50:04.743884 7337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:04.751511 7337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:04.768916 7337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:50:04.770185 7272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38079: connect: Connection refused (error 111)
W20260809 07:50:04.771854 7337 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38079: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38079: connect: Connection refused (error 111)
I20260809 07:50:04.821273 7337 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } attempt: 1
W20260809 07:50:04.823552 7337 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38079: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38079: connect: Connection refused (error 111)
I20260809 07:50:04.891880 7337 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } attempt: 2
W20260809 07:50:04.894301 7337 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38079: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38079: connect: Connection refused (error 111)
I20260809 07:50:04.985605 7337 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } attempt: 3
W20260809 07:50:04.988016 7337 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38079: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38079: connect: Connection refused (error 111)
W20260809 07:50:05.090250 7339 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:05.090684 7339 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:05.090801 7339 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:05.100946 7339 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:05.101032 7339 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:05.101087 7339 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:05.101138 7339 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:05.111194 7339 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:05.112900 7339 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:05.114869 7339 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:05.123000 7350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:05.123303 7349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:05.124022 7352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:05.124790 7339 server_base.cc:1061] running on GCE node
I20260809 07:50:05.125640 7339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:05.127046 7339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:05.128348 7339 hybrid_clock.cc:648] HybridClock initialized: now 1786261805128284 us; error 64 us; skew 500 ppm
I20260809 07:50:05.128765 7339 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:05.130985 7339 webserver.cc:533] Webserver started at http://127.26.37.61:46567/ using document root <none> and password file <none>
I20260809 07:50:05.131292 7337 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } attempt: 4
I20260809 07:50:05.131824 7339 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:05.131986 7339 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:05.132462 7339 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
W20260809 07:50:05.133322 7337 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38079: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38079: connect: Connection refused (error 111)
I20260809 07:50:05.135111 7339 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "7debac8e871a4fcd99f27cd643974303"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-bw88"
I20260809 07:50:05.135964 7339 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "7debac8e871a4fcd99f27cd643974303"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-bw88"
I20260809 07:50:05.141795 7339 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:50:05.145291 7360 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:05.147012 7339 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:50:05.147202 7339 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "7debac8e871a4fcd99f27cd643974303"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-bw88"
I20260809 07:50:05.147440 7339 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:05.165913 7339 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:05.166755 7339 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:05.167016 7339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:05.187969 7339 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:38079
I20260809 07:50:05.187979 7433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:38079 every 8 connection(s)
I20260809 07:50:05.189592 7339 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:50:05.193814 7434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:05.197216 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7339
I20260809 07:50:05.197417 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:50:05.198546 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:35093
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093 with env {}
I20260809 07:50:05.199931 7434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:05.213722 7434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:05.223309 7434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:50:05.224498 7363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
W20260809 07:50:05.226181 7434 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
I20260809 07:50:05.275491 7434 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } attempt: 1
W20260809 07:50:05.277617 7434 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
I20260809 07:50:05.345870 7434 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } attempt: 2
W20260809 07:50:05.348129 7434 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
I20260809 07:50:05.432616 7337 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } attempt: 5
I20260809 07:50:05.439360 7434 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } attempt: 3
I20260809 07:50:05.440335 7337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:50:05.441457 7434 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
W20260809 07:50:05.442471 7337 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
I20260809 07:50:05.493717 7337 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } attempt: 1
W20260809 07:50:05.496063 7337 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
W20260809 07:50:05.547806 7439 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:05.548189 7439 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:05.548336 7439 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:05.556828 7439 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:05.556945 7439 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:05.557058 7439 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:05.557132 7439 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:05.564327 7337 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } attempt: 2
W20260809 07:50:05.566533 7337 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
I20260809 07:50:05.567957 7439 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:35093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:05.569823 7439 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:05.571758 7439 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:05.580217 7446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:05.580899 7447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:05.581363 7449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:05.582424 7439 server_base.cc:1061] running on GCE node
I20260809 07:50:05.583303 7439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:05.584709 7434 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } attempt: 4
I20260809 07:50:05.584848 7439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:05.586114 7439 hybrid_clock.cc:648] HybridClock initialized: now 1786261805586060 us; error 50 us; skew 500 ppm
I20260809 07:50:05.586551 7439 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260809 07:50:05.586864 7434 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111)
I20260809 07:50:05.588847 7439 webserver.cc:533] Webserver started at http://127.26.37.60:41083/ using document root <none> and password file <none>
I20260809 07:50:05.589720 7439 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:05.589866 7439 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:05.590263 7439 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:05.592833 7439 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "fa44fcd815e5420283c10ff8f89ac2d6"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-bw88"
I20260809 07:50:05.593731 7439 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "fa44fcd815e5420283c10ff8f89ac2d6"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-bw88"
I20260809 07:50:05.599354 7439 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:50:05.603132 7456 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:05.604584 7439 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:50:05.604795 7439 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "fa44fcd815e5420283c10ff8f89ac2d6"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-bw88"
I20260809 07:50:05.605041 7439 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:05.624742 7439 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:05.625758 7439 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:05.626067 7439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:05.647168 7439 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:35093
I20260809 07:50:05.647190 7524 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:35093 every 8 connection(s)
I20260809 07:50:05.648874 7439 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:50:05.653669 7525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:05.655295 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7439
I20260809 07:50:05.655529 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:50:05.661579 7525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:05.672763 7337 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } attempt: 3
I20260809 07:50:05.676257 7525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:05.683640 7525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:05.685690 7337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap starting.
I20260809 07:50:05.689754 7337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:05.691133 7337 log.cc:826] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:05.694326 7337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: No bootstrap required, opened a new log
I20260809 07:50:05.695886 7525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Bootstrap starting.
I20260809 07:50:05.700071 7525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:05.701508 7525 log.cc:826] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:05.702366 7337 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:05.702720 7337 raft_consensus.cc:385] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:05.702869 7337 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6de27bda874235b6a67da26180ff6a, State: Initialized, Role: FOLLOWER
I20260809 07:50:05.703586 7337 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:05.704582 7535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:05.704857 7535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:05.704919 7525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: No bootstrap required, opened a new log
I20260809 07:50:05.706043 7337 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:05.713495 7525 raft_consensus.cc:359] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:05.713954 7525 raft_consensus.cc:385] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:05.714136 7525 raft_consensus.cc:740] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa44fcd815e5420283c10ff8f89ac2d6, State: Initialized, Role: FOLLOWER
I20260809 07:50:05.714929 7525 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
W20260809 07:50:05.718802 7552 catalog_manager.cc:1594] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:50:05.718964 7552 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 07:50:05.719214 7277 tablet.cc:2406] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:50:05.719686 7555 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:05.720065 7555 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:05.721803 7525 sys_catalog.cc:565] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:05.718639 7256 master_runner.cc:187] Error getting master registration for 127.26.37.61:38079: OK, instance_id { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" instance_seqno: 1786261805182346 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:50:05.724033 7256 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:50:05.734359 7570 catalog_manager.cc:1594] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:50:05.734618 7570 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:50:05.734397 7439 master_runner.cc:187] Error getting master registration for 127.26.37.61:38079: OK, instance_id { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" instance_seqno: 1786261805182346 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:50:05.741192 7439 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:50:05.860945 7535 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:50:05.861272 7535 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:05.863212 7535 leader_election.cc:290] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7debac8e871a4fcd99f27cd643974303 (127.26.37.61:38079), fa44fcd815e5420283c10ff8f89ac2d6 (127.26.37.60:35093)
I20260809 07:50:05.864230 7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd6de27bda874235b6a67da26180ff6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" is_pre_election: true
I20260809 07:50:05.864826 7495 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6de27bda874235b6a67da26180ff6a in term 0.
I20260809 07:50:05.865496 7274 leader_election.cc:304] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6de27bda874235b6a67da26180ff6a, fa44fcd815e5420283c10ff8f89ac2d6; no voters:
I20260809 07:50:05.865954 7535 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:50:05.866118 7535 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:50:05.866219 7535 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:05.869798 7535 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:05.871516 7535 leader_election.cc:290] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [CANDIDATE]: Term 1 election: Requested vote from peers 7debac8e871a4fcd99f27cd643974303 (127.26.37.61:38079), fa44fcd815e5420283c10ff8f89ac2d6 (127.26.37.60:35093)
I20260809 07:50:05.872025 7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd6de27bda874235b6a67da26180ff6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa44fcd815e5420283c10ff8f89ac2d6"
I20260809 07:50:05.872231 7495 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:05.876006 7495 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6de27bda874235b6a67da26180ff6a in term 1.
I20260809 07:50:05.876587 7274 leader_election.cc:304] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6de27bda874235b6a67da26180ff6a, fa44fcd815e5420283c10ff8f89ac2d6; no voters:
I20260809 07:50:05.877789 7535 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 1 FOLLOWER]: Leader election won for term 1
W20260809 07:50:05.878217 7463 tablet.cc:2406] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:50:05.880944 7535 raft_consensus.cc:697] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 1 LEADER]: Becoming Leader. State: Replica: cd6de27bda874235b6a67da26180ff6a, State: Running, Role: LEADER
I20260809 07:50:05.881592 7535 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:05.886099 7434 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } attempt: 5
I20260809 07:50:05.886791 7572 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd6de27bda874235b6a67da26180ff6a. Latest consensus state: current_term: 1 leader_uuid: "cd6de27bda874235b6a67da26180ff6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:05.887126 7572 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:05.901539 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: Bootstrap starting.
I20260809 07:50:05.901914 7574 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:50:05.904271 7574 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:50:05.906059 7434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:05.907750 7434 log.cc:826] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:05.914228 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: No bootstrap required, opened a new log
I20260809 07:50:05.926059 7434 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:05.926545 7434 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:05.926718 7434 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7debac8e871a4fcd99f27cd643974303, State: Initialized, Role: FOLLOWER
I20260809 07:50:05.927251 7495 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 1 FOLLOWER]: Refusing update from remote peer cd6de27bda874235b6a67da26180ff6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:50:05.927428 7434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:05.928503 7572 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:50:05.934365 7579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:05.934674 7579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:05.936445 7434 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:05.936797 7399 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:05.940646 7399 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 1 FOLLOWER]: Refusing update from remote peer cd6de27bda874235b6a67da26180ff6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:50:05.941738 7400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd6de27bda874235b6a67da26180ff6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7debac8e871a4fcd99f27cd643974303"
I20260809 07:50:05.945524 7402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd6de27bda874235b6a67da26180ff6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7debac8e871a4fcd99f27cd643974303" is_pre_election: true
I20260809 07:50:05.946913 7572 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:50:05.951854 7400 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6de27bda874235b6a67da26180ff6a in term 1.
I20260809 07:50:05.980443 7579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd6de27bda874235b6a67da26180ff6a. Latest consensus state: current_term: 1 leader_uuid: "cd6de27bda874235b6a67da26180ff6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:05.980716 7579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:05.990530 7579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd6de27bda874235b6a67da26180ff6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:05.990826 7579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:05.991464 7572 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd6de27bda874235b6a67da26180ff6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:05.992007 7572 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:05.992365 7589 mvcc.cc:204] Tried to move back new op lower bound from 7316528357064351744 to 7316528356896571392. Current Snapshot: MvccSnapshot[applied={T|T < 7316528357064351744 or (T in {7316528357064351744})}]
I20260809 07:50:05.995445 7574 catalog_manager.cc:1383] Generated new cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:05.995579 7574 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:50:06.012389 7602 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:50:06.012676 7555 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd6de27bda874235b6a67da26180ff6a. Latest consensus state: current_term: 1 leader_uuid: "cd6de27bda874235b6a67da26180ff6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:06.012945 7555 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:06.014787 7572 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd6de27bda874235b6a67da26180ff6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:06.015002 7572 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:06.020696 7584 mvcc.cc:204] Tried to move back new op lower bound from 7316528357064351744 to 7316528356896571392. Current Snapshot: MvccSnapshot[applied={T|T < 7316528357064351744}]
I20260809 07:50:06.025966 7555 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd6de27bda874235b6a67da26180ff6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:06.026191 7555 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:06.051687 7574 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:50:06.054410 7574 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:50:06.077769 7574 catalog_manager.cc:6092] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Generated new TSK 0
I20260809 07:50:06.078959 7574 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:50:06.089385 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:50:06.493912 7604 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:06.494252 7604 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:06.494390 7604 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:06.502650 7604 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:06.502894 7604 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:06.513180 7604 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:06.514910 7604 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:06.516944 7604 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:06.526611 7611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:06.526626 7609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:06.527591 7613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:06.527956 7604 server_base.cc:1061] running on GCE node
I20260809 07:50:06.528527 7604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:06.529419 7604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:06.530652 7604 hybrid_clock.cc:648] HybridClock initialized: now 1786261806530606 us; error 52 us; skew 500 ppm
I20260809 07:50:06.530956 7604 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:06.533034 7604 webserver.cc:533] Webserver started at http://127.26.37.1:35265/ using document root <none> and password file <none>
I20260809 07:50:06.533842 7604 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:06.533990 7604 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:06.534369 7604 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:06.536803 7604 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:06.537688 7604 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:06.543046 7604 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:50:06.546487 7619 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.547824 7604 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:50:06.547953 7604 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:06.548156 7604 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:06.566164 7604 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:06.566993 7604 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:06.567233 7604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:06.568110 7604 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:06.569770 7604 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:06.569856 7604 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.569932 7604 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:06.569988 7604 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.606133 7604 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:32827
I20260809 07:50:06.606191 7754 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:32827 every 8 connection(s)
I20260809 07:50:06.607827 7604 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:06.613490 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7604
I20260809 07:50:06.613720 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:50:06.620317 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:06.657025 7757 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:06.657128 7755 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:06.657387 7758 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:06.657588 7758 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:06.657542 7755 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:06.657543 7757 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:06.658831 7758 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:06.659036 7755 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:06.661504 7481 ts_manager.cc:194] Registered new tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:32827)
I20260809 07:50:06.661604 7387 ts_manager.cc:194] Registered new tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:32827)
I20260809 07:50:06.663030 7757 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:06.664856 7290 ts_manager.cc:194] Registered new tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:32827)
I20260809 07:50:06.667132 7290 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:50527
I20260809 07:50:06.736744 7570 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:06.736861 7570 catalog_manager.cc:1588] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: loading cluster ID for follower catalog manager: success
I20260809 07:50:06.740438 7570 catalog_manager.cc:1610] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: acquiring CA information for follower catalog manager: success
I20260809 07:50:06.742726 7570 catalog_manager.cc:1638] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:50:06.995611 7769 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:06.995980 7769 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:06.996129 7769 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:07.004491 7769 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:07.004706 7769 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:50:07.014845 7769 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:07.016611 7769 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:07.017951 7602 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:07.018088 7602 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: loading cluster ID for follower catalog manager: success
I20260809 07:50:07.018734 7769 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:50:07.022107 7602 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: acquiring CA information for follower catalog manager: success
I20260809 07:50:07.024755 7602 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:50:07.028098 7776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:07.028288 7777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:07.030212 7779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:07.030637 7769 server_base.cc:1061] running on GCE node
I20260809 07:50:07.031193 7769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:07.032186 7769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:07.033465 7769 hybrid_clock.cc:648] HybridClock initialized: now 1786261807033379 us; error 63 us; skew 500 ppm
I20260809 07:50:07.033777 7769 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:07.036006 7769 webserver.cc:533] Webserver started at http://127.26.37.2:37273/ using document root <none> and password file <none>
I20260809 07:50:07.036757 7769 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:07.036929 7769 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:07.037304 7769 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:07.039937 7769 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "babdeb93088b470fb1aebd3d4256779d"
format_stamp: "Formatted at 2026-08-09 07:50:07 on dist-test-slave-bw88"
I20260809 07:50:07.040751 7769 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "babdeb93088b470fb1aebd3d4256779d"
format_stamp: "Formatted at 2026-08-09 07:50:07 on dist-test-slave-bw88"
I20260809 07:50:07.046710 7769 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260809 07:50:07.050082 7787 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.051517 7769 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:50:07.051687 7769 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "babdeb93088b470fb1aebd3d4256779d"
format_stamp: "Formatted at 2026-08-09 07:50:07 on dist-test-slave-bw88"
I20260809 07:50:07.051898 7769 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:07.084630 7769 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:07.085906 7769 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:07.086256 7769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:07.089740 7769 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:07.092331 7769 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:07.092468 7769 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.092583 7769 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:07.092684 7769 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.146167 7769 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:33895
I20260809 07:50:07.146222 7941 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:33895 every 8 connection(s)
I20260809 07:50:07.147670 7769 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:50:07.152740 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7769
I20260809 07:50:07.152987 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:50:07.166599 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:07.181103 7942 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:07.181561 7942 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:07.181599 7945 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:07.181831 7945 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:07.182834 7942 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:07.183524 7945 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:07.184787 7481 ts_manager.cc:194] Registered new tserver with Master: babdeb93088b470fb1aebd3d4256779d (127.26.37.2:33895)
I20260809 07:50:07.185796 7387 ts_manager.cc:194] Registered new tserver with Master: babdeb93088b470fb1aebd3d4256779d (127.26.37.2:33895)
I20260809 07:50:07.188385 7944 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:07.188508 7944 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:07.188974 7944 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:07.189913 7290 ts_manager.cc:194] Registered new tserver with Master: babdeb93088b470fb1aebd3d4256779d (127.26.37.2:33895)
I20260809 07:50:07.190903 7290 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:43999
W20260809 07:50:07.531718 7954 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:07.532012 7954 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:07.532167 7954 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:07.540388 7954 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:07.540585 7954 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:50:07.550609 7954 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:07.552495 7954 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:07.554544 7954 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:07.564538 7961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:07.564790 7960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:07.565904 7963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:07.566256 7954 server_base.cc:1061] running on GCE node
I20260809 07:50:07.566892 7954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:07.567870 7954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:07.569116 7954 hybrid_clock.cc:648] HybridClock initialized: now 1786261807569072 us; error 30 us; skew 500 ppm
I20260809 07:50:07.569547 7954 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:07.571831 7954 webserver.cc:533] Webserver started at http://127.26.37.3:45997/ using document root <none> and password file <none>
I20260809 07:50:07.572690 7954 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:07.572821 7954 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:07.573194 7954 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:07.575682 7954 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "6c669127ee354f9bb5fe0dab438defe4"
format_stamp: "Formatted at 2026-08-09 07:50:07 on dist-test-slave-bw88"
I20260809 07:50:07.576459 7954 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "6c669127ee354f9bb5fe0dab438defe4"
format_stamp: "Formatted at 2026-08-09 07:50:07 on dist-test-slave-bw88"
I20260809 07:50:07.581808 7954 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 07:50:07.585227 7972 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.586591 7954 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:50:07.586771 7954 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "6c669127ee354f9bb5fe0dab438defe4"
format_stamp: "Formatted at 2026-08-09 07:50:07 on dist-test-slave-bw88"
I20260809 07:50:07.587005 7954 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:07.609761 7954 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:07.610702 7954 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:07.611012 7954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:07.612044 7954 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:07.613943 7954 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:07.614054 7954 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.614180 7954 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:07.614272 7954 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.651577 7954 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:38701
I20260809 07:50:07.651631 8119 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:38701 every 8 connection(s)
I20260809 07:50:07.653193 7954 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:50:07.654331 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7954
I20260809 07:50:07.654557 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:50:07.670545 7757 heartbeater.cc:499] Master 127.26.37.62:34301 was elected leader, sending a full tablet report...
I20260809 07:50:07.676147 8121 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:07.676672 8121 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:07.677904 8121 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:07.680197 8120 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:07.680209 7290 ts_manager.cc:194] Registered new tserver with Master: 6c669127ee354f9bb5fe0dab438defe4 (127.26.37.3:38701)
I20260809 07:50:07.680343 8120 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:07.680843 8120 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:07.681370 7290 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:35835
I20260809 07:50:07.682024 8122 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:07.682142 7480 ts_manager.cc:194] Registered new tserver with Master: 6c669127ee354f9bb5fe0dab438defe4 (127.26.37.3:38701)
I20260809 07:50:07.682209 8122 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:07.682775 8122 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:07.683857 7387 ts_manager.cc:194] Registered new tserver with Master: 6c669127ee354f9bb5fe0dab438defe4 (127.26.37.3:38701)
I20260809 07:50:07.688282 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:50:07.696281 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:50:07.696501 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7256
I20260809 07:50:07.697412 7328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:07.808470 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7256
I20260809 07:50:07.836740 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34301
--webserver_interface=127.26.37.62
--webserver_port=43769
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093 with env {}
W20260809 07:50:08.187045 8128 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:08.187409 8128 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:08.187556 8128 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:50:08.193607 7944 heartbeater.cc:499] Master 127.26.37.62:34301 was elected leader, sending a full tablet report...
W20260809 07:50:08.195343 7944 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:34301 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:34301: connect: Connection refused (error 111)
W20260809 07:50:08.196723 8128 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:08.196838 8128 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:08.196954 8128 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:08.197036 8128 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:08.207271 8128 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=43769
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:08.208971 8128 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:08.210929 8128 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:08.219725 8136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:08.219847 8135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:08.220371 8138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:08.221599 8128 server_base.cc:1061] running on GCE node
I20260809 07:50:08.222373 8128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:08.223845 8128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:08.225047 8128 hybrid_clock.cc:648] HybridClock initialized: now 1786261808225000 us; error 36 us; skew 500 ppm
I20260809 07:50:08.225414 8128 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:08.227502 8128 webserver.cc:533] Webserver started at http://127.26.37.62:43769/ using document root <none> and password file <none>
I20260809 07:50:08.228195 8128 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:08.228299 8128 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:08.233701 8128 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:08.236407 8152 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:08.237861 8128 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:08.238080 8128 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cd6de27bda874235b6a67da26180ff6a"
format_stamp: "Formatted at 2026-08-09 07:50:04 on dist-test-slave-bw88"
I20260809 07:50:08.238835 8128 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:08.267128 8128 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:08.267994 8128 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:08.268249 8128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:08.289086 8128 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34301
I20260809 07:50:08.289109 8215 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34301 every 8 connection(s)
I20260809 07:50:08.290838 8128 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:08.298017 8217 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:50:08.300402 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 8128
I20260809 07:50:08.300751 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7604
I20260809 07:50:08.301455 8217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap starting.
I20260809 07:50:08.310092 7748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:08.321110 8217 log.cc:826] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:08.330631 8217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:50:08.331210 8217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap complete.
I20260809 07:50:08.338208 8217 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:08.338943 8217 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6de27bda874235b6a67da26180ff6a, State: Initialized, Role: FOLLOWER
I20260809 07:50:08.339795 8217 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:08.340682 8225 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:08.341015 8225 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:08.341881 8217 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:08.353466 8239 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:08.353725 8239 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: loading cluster ID for follower catalog manager: success
I20260809 07:50:08.359163 8239 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: acquiring CA information for follower catalog manager: success
I20260809 07:50:08.362754 8239 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:50:08.394120 8128 master_runner.cc:264] No leader master found from master cd6de27bda874235b6a67da26180ff6a
I20260809 07:50:08.394307 8128 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:50:08.427744 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7604
I20260809 07:50:08.459438 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:32827
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--webserver_port=35265
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:08.684505 8121 heartbeater.cc:499] Master 127.26.37.62:34301 was elected leader, sending a full tablet report...
I20260809 07:50:08.707430 8175 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6c669127ee354f9bb5fe0dab438defe4" instance_seqno: 1786261807642254) as {username='slave'} at 127.26.37.3:48239; Asking this server to re-register.
I20260809 07:50:08.708667 8121 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:08.710445 8175 ts_manager.cc:194] Registered new tserver with Master: 6c669127ee354f9bb5fe0dab438defe4 (127.26.37.3:38701)
I20260809 07:50:08.809949 8253 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6de27bda874235b6a67da26180ff6a)
I20260809 07:50:08.810240 8253 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:08.812052 8253 leader_election.cc:290] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6de27bda874235b6a67da26180ff6a (127.26.37.62:34301), 7debac8e871a4fcd99f27cd643974303 (127.26.37.61:38079)
I20260809 07:50:08.813226 7399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7debac8e871a4fcd99f27cd643974303" is_pre_election: true
I20260809 07:50:08.821478 8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6de27bda874235b6a67da26180ff6a" is_pre_election: true
I20260809 07:50:08.822180 8187 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa44fcd815e5420283c10ff8f89ac2d6 in term 1.
I20260809 07:50:08.822831 7458 leader_election.cc:304] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6de27bda874235b6a67da26180ff6a, fa44fcd815e5420283c10ff8f89ac2d6; no voters: 7debac8e871a4fcd99f27cd643974303
I20260809 07:50:08.823632 8253 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:50:08.823757 8253 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6de27bda874235b6a67da26180ff6a)
I20260809 07:50:08.823889 8253 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:50:08.827198 8253 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:08.828526 8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6de27bda874235b6a67da26180ff6a"
I20260809 07:50:08.828617 7399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7debac8e871a4fcd99f27cd643974303"
I20260809 07:50:08.828756 8187 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:50:08.829398 8253 leader_election.cc:290] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [CANDIDATE]: Term 2 election: Requested vote from peers cd6de27bda874235b6a67da26180ff6a (127.26.37.62:34301), 7debac8e871a4fcd99f27cd643974303 (127.26.37.61:38079)
I20260809 07:50:08.834978 8187 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa44fcd815e5420283c10ff8f89ac2d6 in term 2.
I20260809 07:50:08.835474 7458 leader_election.cc:304] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6de27bda874235b6a67da26180ff6a, fa44fcd815e5420283c10ff8f89ac2d6; no voters: 7debac8e871a4fcd99f27cd643974303
I20260809 07:50:08.835933 8253 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:50:08.836398 8253 raft_consensus.cc:697] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 2 LEADER]: Becoming Leader. State: Replica: fa44fcd815e5420283c10ff8f89ac2d6, State: Running, Role: LEADER
I20260809 07:50:08.836946 8253 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:08.838969 8256 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa44fcd815e5420283c10ff8f89ac2d6. Latest consensus state: current_term: 2 leader_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:08.839249 8256 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:08.841121 8259 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:50:08.843823 8259 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:50:08.845105 8259 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:08.845278 8259 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:50:08.847033 8259 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:50:08.848160 8259 catalog_manager.cc:6103] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Loaded TSK: 0
I20260809 07:50:08.849300 8259 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260809 07:50:08.855737 8251 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:08.856096 8251 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:08.856231 8251 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:08.864784 8251 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:08.865008 8251 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:08.875137 8251 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:32827
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=35265
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:08.876827 8251 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:08.878818 8251 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:08.888535 8266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:08.888571 8268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:08.889672 8271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:08.890030 8251 server_base.cc:1061] running on GCE node
I20260809 07:50:08.890725 8251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:08.891775 8251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:08.892994 8251 hybrid_clock.cc:648] HybridClock initialized: now 1786261808892938 us; error 51 us; skew 500 ppm
I20260809 07:50:08.893324 8251 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:08.895629 8251 webserver.cc:533] Webserver started at http://127.26.37.1:35265/ using document root <none> and password file <none>
I20260809 07:50:08.896428 8251 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:08.896607 8251 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:08.902197 8251 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:50:08.904980 8279 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:08.906435 8251 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:50:08.906648 8251 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:08.907402 8251 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:08.927241 8251 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:08.928110 8251 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:08.928416 8251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:08.929615 8251 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:08.931342 8251 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:08.931450 8251 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:08.931610 8251 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:08.931689 8251 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:08.967374 8251 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:32827
I20260809 07:50:08.967409 8430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:32827 every 8 connection(s)
I20260809 07:50:08.969079 8251 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:08.974197 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 8251
I20260809 07:50:09.008594 8434 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:09.009024 8434 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:09.010398 8434 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:09.010587 8444 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6de27bda874235b6a67da26180ff6a)
I20260809 07:50:09.010846 8444 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:09.013198 7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7debac8e871a4fcd99f27cd643974303" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" is_pre_election: true
I20260809 07:50:09.013435 8175 ts_manager.cc:194] Registered new tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:32827)
I20260809 07:50:09.016429 8444 leader_election.cc:290] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6de27bda874235b6a67da26180ff6a (127.26.37.62:34301), fa44fcd815e5420283c10ff8f89ac2d6 (127.26.37.60:35093)
I20260809 07:50:09.023360 8435 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:09.023551 8435 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:09.024114 8435 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:09.025615 7387 ts_manager.cc:194] Re-registered known tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:32827)
I20260809 07:50:09.029127 8431 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:09.029256 8431 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:09.029780 8431 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:09.031002 7480 ts_manager.cc:194] Re-registered known tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:32827)
I20260809 07:50:09.033279 7480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:45301
I20260809 07:50:09.034946 8187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7debac8e871a4fcd99f27cd643974303" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6de27bda874235b6a67da26180ff6a" is_pre_election: true
I20260809 07:50:09.035354 8187 raft_consensus.cc:2393] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7debac8e871a4fcd99f27cd643974303 in current term 2: Already voted for candidate fa44fcd815e5420283c10ff8f89ac2d6 in this term.
I20260809 07:50:09.036631 7362 leader_election.cc:304] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7debac8e871a4fcd99f27cd643974303; no voters: cd6de27bda874235b6a67da26180ff6a, fa44fcd815e5420283c10ff8f89ac2d6
I20260809 07:50:09.037122 8444 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:50:09.040314 8444 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:50:09.041715 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 8251
I20260809 07:50:09.057875 8421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:09.143659 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 8251
I20260809 07:50:09.176120 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:32827
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--webserver_port=35265
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260809 07:50:09.203109 7944 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:09.203362 7944 heartbeater.cc:499] Master 127.26.37.62:34301 was elected leader, sending a full tablet report...
I20260809 07:50:09.204419 8175 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "babdeb93088b470fb1aebd3d4256779d" instance_seqno: 1786261807134327) as {username='slave'} at 127.26.37.2:36697; Asking this server to re-register.
I20260809 07:50:09.205081 7944 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:09.206315 8175 ts_manager.cc:194] Registered new tserver with Master: babdeb93088b470fb1aebd3d4256779d (127.26.37.2:33895)
I20260809 07:50:09.260689 8187 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 2 FOLLOWER]: Refusing update from remote peer fa44fcd815e5420283c10ff8f89ac2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:50:09.261678 8256 consensus_queue.cc:1048] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:50:09.265561 8453 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa44fcd815e5420283c10ff8f89ac2d6. Latest consensus state: current_term: 2 leader_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:09.265853 8453 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:09.267004 8256 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:09.267295 8256 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:09.267884 7399 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Refusing update from remote peer fa44fcd815e5420283c10ff8f89ac2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:50:09.268071 8453 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:09.268244 8453 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:09.268615 8253 consensus_queue.cc:1048] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:50:09.272625 8444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa44fcd815e5420283c10ff8f89ac2d6. Latest consensus state: current_term: 2 leader_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:09.272907 8444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:09.272505 8460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:09.273165 8460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:50:09.529491 8450 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:09.529803 8450 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:09.529953 8450 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:50:09.530033 8450 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:09.537906 8450 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:09.538149 8450 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:09.547994 8450 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:32827
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=35265
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:09.549772 8450 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:09.551498 8450 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:09.561066 8467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:09.561043 8468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:09.561849 8471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:09.562552 8450 server_base.cc:1061] running on GCE node
I20260809 07:50:09.563170 8450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:09.564046 8450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:09.565291 8450 hybrid_clock.cc:648] HybridClock initialized: now 1786261809565235 us; error 57 us; skew 500 ppm
I20260809 07:50:09.565613 8450 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:09.567683 8450 webserver.cc:533] Webserver started at http://127.26.37.1:35265/ using document root <none> and password file <none>
I20260809 07:50:09.568439 8450 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:09.568604 8450 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:09.573887 8450 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:50:09.576725 8477 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:09.578162 8450 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:09.578346 8450 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:09.579147 8450 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:09.599659 8450 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:09.600495 8450 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:09.600764 8450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:09.601771 8450 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:09.603384 8450 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:09.603528 8450 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:09.603658 8450 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:09.603760 8450 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:50:09.604135 8450 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:50:09.604243 8450 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:50:09.604332 8450 fault_injection.cc:59] injecting fault for kudu (pid 8450): FLAGS_fault_before_start (process will exit)
I20260809 07:50:09.631381 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:32827
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--webserver_port=35265
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:09.850633 8491 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:09.850903 8491 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: LEADER
W20260809 07:50:09.980351 8488 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:09.980713 8488 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:09.980858 8488 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:09.988684 8488 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:09.988883 8488 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:09.998665 8488 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:32827
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=35265
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:10.000305 8488 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:10.002235 8488 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:10.011760 8498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:10.011726 8499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:10.012622 8501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:10.013043 8488 server_base.cc:1061] running on GCE node
I20260809 07:50:10.013756 8488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:10.014636 8488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:10.015898 8488 hybrid_clock.cc:648] HybridClock initialized: now 1786261810015853 us; error 69 us; skew 500 ppm
I20260809 07:50:10.016237 8488 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:10.018494 8488 webserver.cc:533] Webserver started at http://127.26.37.1:35265/ using document root <none> and password file <none>
I20260809 07:50:10.019359 8488 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:10.019513 8488 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:10.025226 8488 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:50:10.028009 8508 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:10.029250 8488 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:50:10.029434 8488 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:10.030151 8488 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:10.048957 8488 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:10.049813 8488 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:10.050096 8488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:10.050981 8488 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:10.052618 8488 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:10.052716 8488 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:10.052857 8488 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:10.052928 8488 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:10.087146 8488 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:32827
I20260809 07:50:10.087186 8655 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:32827 every 8 connection(s)
I20260809 07:50:10.088729 8488 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:10.095199 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 8488
I20260809 07:50:10.095618 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 8488
I20260809 07:50:10.120872 8656 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:10.121373 8656 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:10.121625 8651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:10.191521 7942 heartbeater.cc:499] Master 127.26.37.60:35093 was elected leader, sending a full tablet report...
I20260809 07:50:10.214090 8658 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:10.214191 8657 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:10.214644 8658 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:10.215116 8657 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:10.215993 8656 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:10.216019 8657 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:10.216665 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 8488
I20260809 07:50:10.216923 8658 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:10.239509 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7769
I20260809 07:50:10.247812 7935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:10.337031 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7769
I20260809 07:50:10.362565 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7954
I20260809 07:50:10.370751 8113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:10.457134 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7954
I20260809 07:50:10.480752 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 8128
I20260809 07:50:10.486922 8211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:10.591368 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 8128
I20260809 07:50:10.618497 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7339
I20260809 07:50:10.619961 7426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:10.734864 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7339
I20260809 07:50:10.743011 7439 master_runner.cc:187] Error getting master registration for 127.26.37.62:34301: Network error: Client connection negotiation failed: client connection to 127.26.37.62:34301: connect: Connection refused (error 111),
I20260809 07:50:10.743247 7439 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:50:10.764636 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7439
I20260809 07:50:10.765988 7520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:10.876024 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7439
Shutting down hive metastore.
2026-08-09T07:50:11Z chronyd exiting
2026-08-09T07:50:11Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:11Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:50:12: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:50:20.953763 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34301
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093 with env {}
W20260809 07:50:21.301457 8909 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:21.301828 8909 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:21.301975 8909 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:21.310223 8909 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:21.310316 8909 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:21.310406 8909 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:21.310499 8909 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:21.320451 8909 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:21.322229 8909 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:21.324041 8909 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:21.332175 8914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:21.332147 8915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:21.332862 8917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:21.333387 8909 server_base.cc:1061] running on GCE node
I20260809 07:50:21.334173 8909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:21.335544 8909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:21.336745 8909 hybrid_clock.cc:648] HybridClock initialized: now 1786261821336689 us; error 55 us; skew 500 ppm
I20260809 07:50:21.337103 8909 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:21.344264 8909 webserver.cc:533] Webserver started at http://127.26.37.62:35695/ using document root <none> and password file <none>
I20260809 07:50:21.345039 8909 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:21.345182 8909 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:21.350792 8909 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 07:50:21.353937 8927 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:21.355376 8909 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:21.355587 8909 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cd6de27bda874235b6a67da26180ff6a"
format_stamp: "Formatted at 2026-08-09 07:50:04 on dist-test-slave-bw88"
I20260809 07:50:21.356349 8909 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:21.399868 8909 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:21.400786 8909 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:21.401109 8909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:21.422407 8909 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34301
I20260809 07:50:21.422439 8992 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34301 every 8 connection(s)
I20260809 07:50:21.424134 8909 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:21.428942 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 8909
I20260809 07:50:21.429147 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:50:21.430398 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38079
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093 with env {}
I20260809 07:50:21.432365 8993 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:50:21.437450 8993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap starting.
I20260809 07:50:21.454989 8993 log.cc:826] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:21.464519 8993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:50:21.465072 8993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap complete.
I20260809 07:50:21.471971 8993 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:21.472707 8993 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6de27bda874235b6a67da26180ff6a, State: Initialized, Role: FOLLOWER
I20260809 07:50:21.473448 8993 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:21.474346 8998 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:21.474653 8998 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:21.475419 8993 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:21.494360 9010 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:21.494503 9010 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: loading cluster ID for follower catalog manager: success
I20260809 07:50:21.497977 9010 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: acquiring CA information for follower catalog manager: success
I20260809 07:50:21.499953 9010 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:50:21.513266 8909 master_runner.cc:187] Error getting master registration for 127.26.37.61:38079: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38079: connect: Connection refused (error 111),
I20260809 07:50:21.519781 8909 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:50:21.847287 8995 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:21.847628 8995 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:21.847790 8995 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:21.857004 8995 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:21.857111 8995 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:21.857203 8995 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:21.857293 8995 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:21.867815 8995 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:21.869645 8995 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:21.871902 8995 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:21.879913 9020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:21.879954 9021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:21.882335 9024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:21.882635 8995 server_base.cc:1061] running on GCE node
I20260809 07:50:21.883680 8995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:21.885695 8995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:21.886969 8995 hybrid_clock.cc:648] HybridClock initialized: now 1786261821886881 us; error 92 us; skew 500 ppm
I20260809 07:50:21.887432 8995 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:21.890431 8995 webserver.cc:533] Webserver started at http://127.26.37.61:44333/ using document root <none> and password file <none>
I20260809 07:50:21.891462 8995 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:21.891652 8995 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:21.899389 8995 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:50:21.903291 9034 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:21.905071 8995 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 07:50:21.905267 8995 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "7debac8e871a4fcd99f27cd643974303"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-bw88"
I20260809 07:50:21.906361 8995 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:21.936021 8995 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:21.937218 8995 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:21.937608 8995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:21.962297 9097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:38079 every 8 connection(s)
I20260809 07:50:21.962302 8995 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:38079
I20260809 07:50:21.964694 8995 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:50:21.968092 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 8995
I20260809 07:50:21.968279 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:50:21.969430 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:35093
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093 with env {}
I20260809 07:50:21.972601 9098 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:50:21.976171 9098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: Bootstrap starting.
I20260809 07:50:22.000028 9098 log.cc:826] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:22.012406 9098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:50:22.013095 9098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: Bootstrap complete.
I20260809 07:50:22.022941 9098 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:22.023916 9098 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7debac8e871a4fcd99f27cd643974303, State: Initialized, Role: FOLLOWER
I20260809 07:50:22.024752 9098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:22.025722 9104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:22.026027 9104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:22.027012 9098 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:22.037726 9118 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:22.037904 9118 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: loading cluster ID for follower catalog manager: success
I20260809 07:50:22.042086 9118 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: acquiring CA information for follower catalog manager: success
I20260809 07:50:22.044658 9118 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:50:22.079197 8995 master_runner.cc:187] Error getting master registration for 127.26.37.60:35093: Network error: Client connection negotiation failed: client connection to 127.26.37.60:35093: connect: Connection refused (error 111),
I20260809 07:50:22.085001 8995 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:50:22.360708 9101 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:22.361090 9101 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:22.361239 9101 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:22.370158 9101 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:22.370281 9101 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:22.370376 9101 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:22.370484 9101 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:22.381302 9101 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:35093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:22.383095 9101 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:22.385027 9101 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:22.393504 9132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:22.393534 9131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:22.394582 9137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:22.395583 9101 server_base.cc:1061] running on GCE node
I20260809 07:50:22.396369 9101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:22.397979 9101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:22.399267 9101 hybrid_clock.cc:648] HybridClock initialized: now 1786261822399211 us; error 38 us; skew 500 ppm
I20260809 07:50:22.399703 9101 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:22.402850 9101 webserver.cc:533] Webserver started at http://127.26.37.60:34467/ using document root <none> and password file <none>
I20260809 07:50:22.403855 9101 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:22.403998 9101 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:22.411336 9101 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:50:22.415162 9147 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:22.416893 9101 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:50:22.417071 9101 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "fa44fcd815e5420283c10ff8f89ac2d6"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-bw88"
I20260809 07:50:22.418104 9101 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:22.438966 9101 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:22.440040 9101 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:22.440379 9101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:22.463291 9101 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:35093
I20260809 07:50:22.463308 9211 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:35093 every 8 connection(s)
I20260809 07:50:22.465000 9101 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:50:22.470896 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9101
I20260809 07:50:22.471130 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:50:22.473721 9212 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:50:22.478734 9212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Bootstrap starting.
I20260809 07:50:22.499060 9212 log.cc:826] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:22.508100 9212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:50:22.508688 9212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: Bootstrap complete.
I20260809 07:50:22.515240 9212 raft_consensus.cc:359] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:22.516301 9212 raft_consensus.cc:740] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa44fcd815e5420283c10ff8f89ac2d6, State: Initialized, Role: FOLLOWER
I20260809 07:50:22.516939 9212 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:22.517812 9219 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:22.518074 9219 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:22.518885 9212 sys_catalog.cc:565] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:22.530393 9231 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:22.530501 9231 catalog_manager.cc:1588] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: loading cluster ID for follower catalog manager: success
I20260809 07:50:22.534015 9231 catalog_manager.cc:1610] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: acquiring CA information for follower catalog manager: success
I20260809 07:50:22.536608 9231 catalog_manager.cc:1638] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:50:22.557397 9101 master_runner.cc:264] No leader master found from master fa44fcd815e5420283c10ff8f89ac2d6
I20260809 07:50:22.557622 9101 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:50:23.269461 9235 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:50:23.269776 9235 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:23.271688 9235 leader_election.cc:290] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6de27bda874235b6a67da26180ff6a (127.26.37.62:34301), fa44fcd815e5420283c10ff8f89ac2d6 (127.26.37.60:35093)
I20260809 07:50:23.272367 8961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7debac8e871a4fcd99f27cd643974303" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6de27bda874235b6a67da26180ff6a" is_pre_election: true
I20260809 07:50:23.272924 8961 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7debac8e871a4fcd99f27cd643974303 in term 2.
I20260809 07:50:23.273626 9036 leader_election.cc:304] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7debac8e871a4fcd99f27cd643974303, cd6de27bda874235b6a67da26180ff6a; no voters:
I20260809 07:50:23.274094 9235 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:50:23.274250 9235 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:50:23.274367 9235 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:50:23.278550 9235 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:23.278880 9183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7debac8e871a4fcd99f27cd643974303" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" is_pre_election: true
I20260809 07:50:23.279354 9235 leader_election.cc:290] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [CANDIDATE]: Term 3 election: Requested vote from peers cd6de27bda874235b6a67da26180ff6a (127.26.37.62:34301), fa44fcd815e5420283c10ff8f89ac2d6 (127.26.37.60:35093)
I20260809 07:50:23.279371 9183 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7debac8e871a4fcd99f27cd643974303 in term 2.
I20260809 07:50:23.279717 8961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7debac8e871a4fcd99f27cd643974303" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6de27bda874235b6a67da26180ff6a"
I20260809 07:50:23.279917 8961 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:50:23.280071 9182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7debac8e871a4fcd99f27cd643974303" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fa44fcd815e5420283c10ff8f89ac2d6"
I20260809 07:50:23.280295 9182 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:50:23.285420 9182 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7debac8e871a4fcd99f27cd643974303 in term 3.
I20260809 07:50:23.285420 8961 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7debac8e871a4fcd99f27cd643974303 in term 3.
I20260809 07:50:23.285879 9037 leader_election.cc:304] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7debac8e871a4fcd99f27cd643974303, fa44fcd815e5420283c10ff8f89ac2d6; no voters:
I20260809 07:50:23.286181 9235 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:50:23.286603 9235 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [term 3 LEADER]: Becoming Leader. State: Replica: 7debac8e871a4fcd99f27cd643974303, State: Running, Role: LEADER
I20260809 07:50:23.287083 9235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:23.289772 9239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7debac8e871a4fcd99f27cd643974303. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:23.290046 9239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:23.290705 9242 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:50:23.292711 9242 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:50:23.293586 9242 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:23.293678 9242 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:50:23.294777 9242 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:50:23.295548 9242 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303: Loaded TSK: 0
I20260809 07:50:23.296391 9242 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:50:23.308334 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:50:23.658437 9243 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:23.658835 9243 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:23.658979 9243 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:23.667645 9243 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:23.667927 9243 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:23.678155 9243 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:23.679939 9243 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:23.681857 9243 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:23.692080 9250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:23.692155 9253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:23.692160 9249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:23.693224 9243 server_base.cc:1061] running on GCE node
I20260809 07:50:23.693926 9243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:23.694813 9243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:23.696024 9243 hybrid_clock.cc:648] HybridClock initialized: now 1786261823695980 us; error 45 us; skew 500 ppm
I20260809 07:50:23.696373 9243 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:23.698565 9243 webserver.cc:533] Webserver started at http://127.26.37.1:38711/ using document root <none> and password file <none>
I20260809 07:50:23.699365 9243 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:23.699515 9243 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:23.704948 9243 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:50:23.707834 9260 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:23.709200 9243 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:50:23.709386 9243 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:23.710098 9243 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:23.725934 9243 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:23.726773 9243 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:23.727074 9243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:23.728019 9243 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:23.729879 9243 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:23.729959 9243 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:23.730101 9243 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:23.730188 9243 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:23.766412 9243 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35691
I20260809 07:50:23.766456 9395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35691 every 8 connection(s)
I20260809 07:50:23.768057 9243 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:23.774040 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9243
I20260809 07:50:23.774308 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:50:23.780905 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:23.806797 9182 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [term 3 FOLLOWER]: Refusing update from remote peer 7debac8e871a4fcd99f27cd643974303: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:50:23.808187 9239 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:50:23.813514 8961 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 3 FOLLOWER]: Refusing update from remote peer 7debac8e871a4fcd99f27cd643974303: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:50:23.814615 9239 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:50:23.815994 9397 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:23.816386 9397 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:23.817526 9397 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:23.818522 9396 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:23.818725 9396 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:23.819182 9396 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:23.821676 9169 ts_manager.cc:194] Registered new tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:35691)
I20260809 07:50:23.822567 9398 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:23.822748 9398 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:23.823293 9398 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:23.824956 9055 ts_manager.cc:194] Registered new tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:35691)
I20260809 07:50:23.826261 9406 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7debac8e871a4fcd99f27cd643974303. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:23.826644 9406 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:23.826807 9055 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:52247
I20260809 07:50:23.835996 9406 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:23.836210 9406 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa44fcd815e5420283c10ff8f89ac2d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:23.837970 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7debac8e871a4fcd99f27cd643974303. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:23.838289 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:23.839092 8949 ts_manager.cc:194] Registered new tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:35691)
I20260809 07:50:23.840641 9410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:23.840813 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:23.841012 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:23.841059 9415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:23.841393 9415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:23.841303 9410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: LEADER
W20260809 07:50:24.164682 9405 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:24.164986 9405 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:24.165145 9405 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:24.173672 9405 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:24.173903 9405 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:50:24.183939 9405 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:24.185612 9405 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:24.187570 9405 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:24.196980 9425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:24.197043 9424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:24.198539 9427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:24.199553 9405 server_base.cc:1061] running on GCE node
I20260809 07:50:24.200167 9405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:24.201076 9405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:24.202340 9405 hybrid_clock.cc:648] HybridClock initialized: now 1786261824202265 us; error 66 us; skew 500 ppm
I20260809 07:50:24.202669 9405 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:24.204880 9405 webserver.cc:533] Webserver started at http://127.26.37.2:41567/ using document root <none> and password file <none>
I20260809 07:50:24.205928 9405 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:24.206097 9405 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:24.216428 9405 fs_manager.cc:714] Time spent opening directory manager: real 0.007s user 0.003s sys 0.006s
I20260809 07:50:24.219544 9435 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:24.221131 9405 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:50:24.221401 9405 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "babdeb93088b470fb1aebd3d4256779d"
format_stamp: "Formatted at 2026-08-09 07:50:07 on dist-test-slave-bw88"
I20260809 07:50:24.222240 9405 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:24.254935 9405 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:24.255898 9405 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:24.256239 9405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:24.257246 9405 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:24.259096 9405 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:24.259204 9405 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:24.259353 9405 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:24.259441 9405 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:24.294094 9405 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:43099
I20260809 07:50:24.294142 9570 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:43099 every 8 connection(s)
I20260809 07:50:24.295581 9405 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:50:24.303236 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9405
I20260809 07:50:24.303429 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:50:24.308419 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:24.320029 9573 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:24.320125 9571 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:24.320149 9574 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:24.320537 9571 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:24.320541 9573 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:24.320544 9574 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:24.321888 9571 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:24.321938 9574 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:24.322391 9573 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:24.323737 9055 ts_manager.cc:194] Registered new tserver with Master: babdeb93088b470fb1aebd3d4256779d (127.26.37.2:43099)
I20260809 07:50:24.323819 9169 ts_manager.cc:194] Registered new tserver with Master: babdeb93088b470fb1aebd3d4256779d (127.26.37.2:43099)
I20260809 07:50:24.323869 8949 ts_manager.cc:194] Registered new tserver with Master: babdeb93088b470fb1aebd3d4256779d (127.26.37.2:43099)
I20260809 07:50:24.324846 9055 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:52447
W20260809 07:50:24.663761 9580 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:24.664172 9580 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:24.664325 9580 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:24.672771 9580 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:24.673026 9580 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:50:24.684026 9580 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:24.686000 9580 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:24.687839 9580 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:24.697741 9589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:24.698275 9590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:24.698711 9592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:24.699375 9580 server_base.cc:1061] running on GCE node
I20260809 07:50:24.700053 9580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:24.701027 9580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:24.702251 9580 hybrid_clock.cc:648] HybridClock initialized: now 1786261824702207 us; error 29 us; skew 500 ppm
I20260809 07:50:24.702606 9580 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:24.704844 9580 webserver.cc:533] Webserver started at http://127.26.37.3:46787/ using document root <none> and password file <none>
I20260809 07:50:24.705717 9580 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:24.705876 9580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:24.711668 9580 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 07:50:24.714576 9598 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:24.715955 9580 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:50:24.716140 9580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "6c669127ee354f9bb5fe0dab438defe4"
format_stamp: "Formatted at 2026-08-09 07:50:07 on dist-test-slave-bw88"
I20260809 07:50:24.716879 9580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:24.744383 9580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:24.745368 9580 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:24.745673 9580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:24.746754 9580 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:24.748538 9580 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:24.748647 9580 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:24.748798 9580 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:24.748886 9580 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:24.786330 9580 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:40645
I20260809 07:50:24.786391 9732 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:40645 every 8 connection(s)
I20260809 07:50:24.787941 9580 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:50:24.788628 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9580
I20260809 07:50:24.788878 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:50:24.807917 9735 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:24.808382 9735 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:24.809645 9735 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:24.810105 9734 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:24.810245 9734 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:24.810678 9734 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:24.811436 9733 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:24.811471 9055 ts_manager.cc:194] Registered new tserver with Master: 6c669127ee354f9bb5fe0dab438defe4 (127.26.37.3:40645)
I20260809 07:50:24.811591 9733 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:24.812027 8949 ts_manager.cc:194] Registered new tserver with Master: 6c669127ee354f9bb5fe0dab438defe4 (127.26.37.3:40645)
I20260809 07:50:24.812137 9733 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:24.812567 9055 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:36917
I20260809 07:50:24.813289 9169 ts_manager.cc:194] Registered new tserver with Master: 6c669127ee354f9bb5fe0dab438defe4 (127.26.37.3:40645)
I20260809 07:50:24.817849 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:50:24.825932 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:50:24.826097 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 8909
I20260809 07:50:24.826993 8988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:24.829687 9398 heartbeater.cc:499] Master 127.26.37.61:38079 was elected leader, sending a full tablet report...
I20260809 07:50:24.936305 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 8909
I20260809 07:50:24.964363 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34301
--webserver_interface=127.26.37.62
--webserver_port=35695
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093 with env {}
W20260809 07:50:25.244843 9036 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:34301: connect: Connection refused (error 111)
W20260809 07:50:25.246507 9036 consensus_peers.cc:597] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 -> Peer cd6de27bda874235b6a67da26180ff6a (127.26.37.62:34301): Couldn't send request to peer cd6de27bda874235b6a67da26180ff6a. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:34301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:50:25.320359 9742 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:25.320678 9742 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:25.320752 9742 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:25.327431 9573 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:34301 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:34301: connect: Connection refused (error 111)
I20260809 07:50:25.327579 9574 heartbeater.cc:499] Master 127.26.37.61:38079 was elected leader, sending a full tablet report...
W20260809 07:50:25.330299 9742 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:25.330384 9742 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:25.330470 9742 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:25.330554 9742 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:25.340523 9742 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=35695
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:25.342206 9742 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:25.343977 9742 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:25.352051 9752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:25.352110 9751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:25.352825 9756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:25.353399 9742 server_base.cc:1061] running on GCE node
I20260809 07:50:25.354209 9742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:25.355635 9742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:25.356912 9742 hybrid_clock.cc:648] HybridClock initialized: now 1786261825356812 us; error 78 us; skew 500 ppm
I20260809 07:50:25.357299 9742 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:25.359730 9742 webserver.cc:533] Webserver started at http://127.26.37.62:35695/ using document root <none> and password file <none>
I20260809 07:50:25.360536 9742 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:25.360687 9742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:25.366786 9742 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:50:25.369911 9765 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:25.371253 9742 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:50:25.371461 9742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cd6de27bda874235b6a67da26180ff6a"
format_stamp: "Formatted at 2026-08-09 07:50:04 on dist-test-slave-bw88"
I20260809 07:50:25.372182 9742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:25.387826 9742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:25.388710 9742 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:25.388942 9742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:25.408931 9742 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34301
I20260809 07:50:25.408931 9833 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34301 every 8 connection(s)
I20260809 07:50:25.410612 9742 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:25.417604 9834 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:50:25.419050 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9742
I20260809 07:50:25.419355 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9243
I20260809 07:50:25.421664 9834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap starting.
I20260809 07:50:25.429273 9390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:25.440578 9834 log.cc:826] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:25.450103 9834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:50:25.450646 9834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: Bootstrap complete.
I20260809 07:50:25.457120 9834 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:25.457962 9834 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6de27bda874235b6a67da26180ff6a, State: Initialized, Role: FOLLOWER
I20260809 07:50:25.458738 9834 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } }
I20260809 07:50:25.459614 9840 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:25.459892 9840 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:25.460649 9834 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:25.480839 9853 catalog_manager.cc:1295] Loaded cluster ID: 5933088e17fa468aa014b0fd556586b3
I20260809 07:50:25.481048 9853 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: loading cluster ID for follower catalog manager: success
I20260809 07:50:25.485689 9853 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: acquiring CA information for follower catalog manager: success
I20260809 07:50:25.488335 9853 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:50:25.536190 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9243
I20260809 07:50:25.562428 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35691
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--webserver_port=38711
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:25.815292 9735 heartbeater.cc:499] Master 127.26.37.61:38079 was elected leader, sending a full tablet report...
I20260809 07:50:25.827348 9788 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6c669127ee354f9bb5fe0dab438defe4" instance_seqno: 1786261824777653) as {username='slave'} at 127.26.37.3:58875; Asking this server to re-register.
I20260809 07:50:25.828217 9734 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:25.830086 9788 ts_manager.cc:194] Registered new tserver with Master: 6c669127ee354f9bb5fe0dab438defe4 (127.26.37.3:40645)
I20260809 07:50:25.868321 9581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:25.868621 9581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:25.880877 9840 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7debac8e871a4fcd99f27cd643974303. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:25.881175 9840 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd6de27bda874235b6a67da26180ff6a [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:50:25.933396 9859 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:25.933705 9859 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:25.933851 9859 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:25.942844 9859 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:25.943027 9859 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:25.953975 9859 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35691
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38711
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:25.955741 9859 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:25.957710 9859 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:25.968086 9871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:25.968514 9872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:25.970543 9874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:25.970968 9859 server_base.cc:1061] running on GCE node
I20260809 07:50:25.971591 9859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:25.972566 9859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:25.973822 9859 hybrid_clock.cc:648] HybridClock initialized: now 1786261825973778 us; error 47 us; skew 500 ppm
I20260809 07:50:25.974136 9859 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:25.976303 9859 webserver.cc:533] Webserver started at http://127.26.37.1:38711/ using document root <none> and password file <none>
I20260809 07:50:25.977130 9859 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:25.977288 9859 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:25.982776 9859 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 07:50:25.985404 9882 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:25.986735 9859 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:50:25.986917 9859 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:25.987718 9859 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:26.018790 9859 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:26.019724 9859 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:26.020094 9859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:26.021075 9859 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:26.023214 9859 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:26.023320 9859 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:26.023478 9859 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:26.023558 9859 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:26.059792 9859 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35691
I20260809 07:50:26.059837 10039 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35691 every 8 connection(s)
I20260809 07:50:26.061501 9859 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:26.070973 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9859
I20260809 07:50:26.095551 10040 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:26.096012 10040 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:26.097064 10040 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:26.099196 10042 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:26.099340 10042 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:26.099651 9169 ts_manager.cc:194] Re-registered known tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:35691)
I20260809 07:50:26.099844 10042 heartbeater.cc:507] Master 127.26.37.62:34301 requested a full tablet report, sending...
I20260809 07:50:26.100287 10043 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:26.100412 10043 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:26.100853 10043 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:26.101714 9788 ts_manager.cc:194] Registered new tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:35691)
I20260809 07:50:26.102085 9055 ts_manager.cc:194] Re-registered known tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:35691)
I20260809 07:50:26.103176 9055 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:45369
I20260809 07:50:26.193717 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9859
I20260809 07:50:26.203789 10033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:26.280161 9861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7debac8e871a4fcd99f27cd643974303" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd6de27bda874235b6a67da26180ff6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34301 } } peers { permanent_uuid: "7debac8e871a4fcd99f27cd643974303" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38079 } } peers { permanent_uuid: "fa44fcd815e5420283c10ff8f89ac2d6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 35093 } } }
I20260809 07:50:26.280524 9861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debac8e871a4fcd99f27cd643974303 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:26.290760 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9859
I20260809 07:50:26.316644 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35691
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--webserver_port=38711
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260809 07:50:26.335672 9573 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:26.336740 9788 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "babdeb93088b470fb1aebd3d4256779d" instance_seqno: 1786261824286456) as {username='slave'} at 127.26.37.2:58519; Asking this server to re-register.
I20260809 07:50:26.337299 9573 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:26.338415 9788 ts_manager.cc:194] Registered new tserver with Master: babdeb93088b470fb1aebd3d4256779d (127.26.37.2:43099)
W20260809 07:50:26.668359 10058 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:26.668736 10058 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:26.668883 10058 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:50:26.669008 10058 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:26.677877 10058 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:26.678123 10058 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:26.687711 10058 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35691
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38711
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:26.689424 10058 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:26.691254 10058 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:26.701097 10065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:26.701119 10066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:26.702160 10068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:26.703450 10058 server_base.cc:1061] running on GCE node
I20260809 07:50:26.704074 10058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:26.704967 10058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:26.706238 10058 hybrid_clock.cc:648] HybridClock initialized: now 1786261826706152 us; error 59 us; skew 500 ppm
I20260809 07:50:26.706588 10058 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:26.708766 10058 webserver.cc:533] Webserver started at http://127.26.37.1:38711/ using document root <none> and password file <none>
I20260809 07:50:26.709623 10058 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:26.709787 10058 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:26.715504 10058 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:50:26.718323 10078 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:26.719712 10058 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:26.719892 10058 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:26.720715 10058 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:26.738301 10058 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:26.739167 10058 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:26.739454 10058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:26.740403 10058 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:26.742177 10058 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:26.742308 10058 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:26.742448 10058 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:26.742550 10058 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:50:26.742897 10058 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:50:26.743017 10058 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:50:26.743111 10058 fault_injection.cc:59] injecting fault for kudu (pid 10058): FLAGS_fault_before_start (process will exit)
I20260809 07:50:26.773334 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35691
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--webserver_port=38711
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:50:27.131059 10089 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:27.131420 10089 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:27.131588 10089 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:27.139861 10089 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:27.140079 10089 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:27.150169 10089 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35691
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38711
--tserver_master_addrs=127.26.37.62:34301,127.26.37.61:38079,127.26.37.60:35093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:27.151897 10089 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:27.153803 10089 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:27.163528 10098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:27.163546 10097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:27.163990 10101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:27.170075 10089 server_base.cc:1061] running on GCE node
I20260809 07:50:27.170713 10089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:27.171746 10089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:27.172960 10089 hybrid_clock.cc:648] HybridClock initialized: now 1786261827172891 us; error 65 us; skew 500 ppm
I20260809 07:50:27.173302 10089 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:27.175722 10089 webserver.cc:533] Webserver started at http://127.26.37.1:38711/ using document root <none> and password file <none>
I20260809 07:50:27.176460 10089 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:27.176617 10089 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:27.182282 10089 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:50:27.185040 10107 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:27.186579 10089 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:50:27.186791 10089 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "1725271a46c7422d9018bfe3054bc9c2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-bw88"
I20260809 07:50:27.187616 10089 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:27.214967 10089 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:27.215996 10089 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:27.216347 10089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:27.217567 10089 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:27.219550 10089 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:27.219669 10089 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:27.219833 10089 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:27.219938 10089 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:27.259819 10089 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35691
I20260809 07:50:27.259868 10247 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35691 every 8 connection(s)
I20260809 07:50:27.261561 10089 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:27.268674 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 10089
I20260809 07:50:27.269070 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 10089
I20260809 07:50:27.296258 10250 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38079
I20260809 07:50:27.296646 10250 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:27.297405 10248 heartbeater.cc:344] Connected to a master server at 127.26.37.60:35093
I20260809 07:50:27.297531 10248 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:27.297724 10243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:27.297835 10250 heartbeater.cc:507] Master 127.26.37.61:38079 requested a full tablet report, sending...
I20260809 07:50:27.386181 10249 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34301
I20260809 07:50:27.387137 10249 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:27.386902 10248 heartbeater.cc:507] Master 127.26.37.60:35093 requested a full tablet report, sending...
I20260809 07:50:27.387957 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 10089
I20260809 07:50:27.388255 9055 ts_manager.cc:194] Re-registered known tserver with Master: 1725271a46c7422d9018bfe3054bc9c2 (127.26.37.1:35691)
I20260809 07:50:27.389273 9055 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:35025
W20260809 07:50:27.412205 9038 connection.cc:573] server connection from 127.26.37.1:35025 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:50:27.412387 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9405
I20260809 07:50:27.421173 9566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:27.509599 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9405
I20260809 07:50:27.535893 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9580
I20260809 07:50:27.544164 9725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:27.631848 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9580
I20260809 07:50:27.655404 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9742
I20260809 07:50:27.661725 9825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:27.766078 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9742
I20260809 07:50:27.793010 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 8995
I20260809 07:50:27.794412 9093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:27.906810 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 8995
W20260809 07:50:27.934827 9152 connection.cc:573] server connection from 127.0.0.1:46122 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:50:27.935321 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9101
I20260809 07:50:27.936949 9207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:28.045259 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9101
Shutting down hive metastore.
2026-08-09T07:50:28Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/4 (40283 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/5
2026-08-09T07:50:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:28Z Disabled control of system clock
2026-08-09T07:50:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:28Z Disabled control of system clock
2026-08-09T07:50:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:28Z Disabled control of system clock
2026-08-09T07:50:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:28Z Disabled control of system clock
2026-08-09T07:50:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:28Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:50:35: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:50:44.875056 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39655
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563 with env {}
W20260809 07:50:45.229547 10652 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:45.229919 10652 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:45.230049 10652 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:45.238442 10652 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:45.238575 10652 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:45.238699 10652 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:45.238785 10652 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:45.248875 10652 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:45.250622 10652 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:45.252560 10652 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:45.261468 10661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:45.261502 10664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:45.261759 10659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:45.262470 10652 server_base.cc:1061] running on GCE node
I20260809 07:50:45.263288 10652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:45.264945 10652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:45.266254 10652 hybrid_clock.cc:648] HybridClock initialized: now 1786261845266099 us; error 126 us; skew 500 ppm
I20260809 07:50:45.266614 10652 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:45.268718 10652 webserver.cc:533] Webserver started at http://127.26.37.62:42013/ using document root <none> and password file <none>
I20260809 07:50:45.269534 10652 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:45.269671 10652 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:45.270053 10652 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:45.272423 10652 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "03c6db696c2944eead26873a78123531"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:50:45.273283 10652 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "03c6db696c2944eead26873a78123531"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:50:45.278941 10652 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 07:50:45.282697 10670 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:45.284159 10652 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:50:45.284349 10652 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "03c6db696c2944eead26873a78123531"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:50:45.284584 10652 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:45.307629 10652 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:45.308521 10652 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:45.308813 10652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:45.328655 10734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39655 every 8 connection(s)
I20260809 07:50:45.328680 10652 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39655
I20260809 07:50:45.330374 10652 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:45.334741 10735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:45.340538 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 10652
I20260809 07:50:45.340771 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:50:45.340751 10735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:45.342372 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:44571
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563 with env {}
I20260809 07:50:45.357275 10735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:50:45.358527 10675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:44571: connect: Connection refused (error 111)
W20260809 07:50:45.360211 10735 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:44571: Network error: Client connection negotiation failed: client connection to 127.26.37.61:44571: connect: Connection refused (error 111)
I20260809 07:50:45.409577 10735 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } attempt: 1
W20260809 07:50:45.411796 10735 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:44571: Network error: Client connection negotiation failed: client connection to 127.26.37.61:44571: connect: Connection refused (error 111)
I20260809 07:50:45.480114 10735 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } attempt: 2
W20260809 07:50:45.482374 10735 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:44571: Network error: Client connection negotiation failed: client connection to 127.26.37.61:44571: connect: Connection refused (error 111)
I20260809 07:50:45.573716 10735 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } attempt: 3
W20260809 07:50:45.575835 10735 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:44571: Network error: Client connection negotiation failed: client connection to 127.26.37.61:44571: connect: Connection refused (error 111)
W20260809 07:50:45.689222 10742 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:45.689641 10742 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:45.689769 10742 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:45.697826 10742 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:45.697901 10742 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:45.697978 10742 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:45.698026 10742 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:45.707835 10742 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:45.709527 10742 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:45.711268 10742 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:50:45.719120 10735 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } attempt: 4
W20260809 07:50:45.719601 10751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:45.720458 10749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:45.720837 10753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:45.721570 10735 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:44571: Network error: Client connection negotiation failed: client connection to 127.26.37.61:44571: connect: Connection refused (error 111)
I20260809 07:50:45.721657 10742 server_base.cc:1061] running on GCE node
I20260809 07:50:45.722417 10742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:45.723894 10742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:45.725183 10742 hybrid_clock.cc:648] HybridClock initialized: now 1786261845725041 us; error 102 us; skew 500 ppm
I20260809 07:50:45.725621 10742 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:45.727893 10742 webserver.cc:533] Webserver started at http://127.26.37.61:38343/ using document root <none> and password file <none>
I20260809 07:50:45.728694 10742 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:45.728858 10742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:45.729260 10742 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:45.732414 10742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "0819118351f44d1ca385876a50b18a95"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:50:45.733284 10742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "0819118351f44d1ca385876a50b18a95"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:50:45.738651 10742 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 07:50:45.742094 10761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:45.743484 10742 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:50:45.743664 10742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "0819118351f44d1ca385876a50b18a95"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:50:45.743871 10742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:45.764631 10742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:45.765483 10742 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:45.765784 10742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:45.786718 10742 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:44571
I20260809 07:50:45.786744 10826 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:44571 every 8 connection(s)
I20260809 07:50:45.788203 10742 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:50:45.790192 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 10742
I20260809 07:50:45.790418 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:50:45.791981 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37563
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563 with env {}
I20260809 07:50:45.793901 10827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:45.802896 10827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:45.816565 10827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:45.826355 10827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:50:45.827445 10765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
W20260809 07:50:45.829038 10827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
I20260809 07:50:45.878346 10827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } attempt: 1
W20260809 07:50:45.880393 10827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
I20260809 07:50:45.948650 10827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } attempt: 2
W20260809 07:50:45.950757 10827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
I20260809 07:50:46.020838 10735 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } attempt: 5
I20260809 07:50:46.027525 10735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:50:46.029558 10735 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
I20260809 07:50:46.041985 10827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } attempt: 3
W20260809 07:50:46.044272 10827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
I20260809 07:50:46.080771 10735 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } attempt: 1
W20260809 07:50:46.082887 10735 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
W20260809 07:50:46.147276 10830 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:46.147650 10830 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:46.147810 10830 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:50:46.151087 10735 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } attempt: 2
W20260809 07:50:46.153054 10735 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
W20260809 07:50:46.156769 10830 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:46.156891 10830 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:46.156986 10830 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:46.157040 10830 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:46.167173 10830 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:46.168808 10830 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:46.170674 10830 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:46.178440 10841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:46.178678 10842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:46.179174 10844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:46.180058 10830 server_base.cc:1061] running on GCE node
I20260809 07:50:46.180850 10830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:46.182502 10830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:46.183701 10830 hybrid_clock.cc:648] HybridClock initialized: now 1786261846183576 us; error 93 us; skew 500 ppm
I20260809 07:50:46.184079 10830 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:46.186462 10830 webserver.cc:533] Webserver started at http://127.26.37.60:40701/ using document root <none> and password file <none>
I20260809 07:50:46.187166 10830 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:46.187294 10830 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:46.187496 10827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } attempt: 4
I20260809 07:50:46.187753 10830 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
W20260809 07:50:46.189522 10827 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111)
I20260809 07:50:46.190708 10830 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "ef94549942544e96b34eb069d6e3f8f6"
format_stamp: "Formatted at 2026-08-09 07:50:46 on dist-test-slave-bw88"
I20260809 07:50:46.191450 10830 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "ef94549942544e96b34eb069d6e3f8f6"
format_stamp: "Formatted at 2026-08-09 07:50:46 on dist-test-slave-bw88"
I20260809 07:50:46.196676 10830 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:50:46.200013 10851 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:46.201519 10830 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:46.201696 10830 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "ef94549942544e96b34eb069d6e3f8f6"
format_stamp: "Formatted at 2026-08-09 07:50:46 on dist-test-slave-bw88"
I20260809 07:50:46.201925 10830 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:46.224388 10830 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:46.225184 10830 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:46.225535 10830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:46.246096 10830 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:37563
I20260809 07:50:46.246114 10922 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:37563 every 8 connection(s)
I20260809 07:50:46.247499 10830 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:50:46.248749 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 10830
I20260809 07:50:46.248982 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:50:46.252666 10924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:46.259244 10735 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } attempt: 3
I20260809 07:50:46.259475 10924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:46.273078 10924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:46.276590 10735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap starting.
I20260809 07:50:46.280061 10924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:50:46.280808 10735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:46.282281 10735 log.cc:826] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:46.288110 10735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: No bootstrap required, opened a new log
I20260809 07:50:46.292614 10924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Bootstrap starting.
I20260809 07:50:46.295673 10735 raft_consensus.cc:359] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:46.295989 10735 raft_consensus.cc:385] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:46.296129 10735 raft_consensus.cc:740] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c6db696c2944eead26873a78123531, State: Initialized, Role: FOLLOWER
I20260809 07:50:46.296557 10924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:46.296813 10735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:46.297991 10924 log.cc:826] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:46.297915 10934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.298322 10934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:46.299218 10735 sys_catalog.cc:565] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:46.300987 10924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: No bootstrap required, opened a new log
I20260809 07:50:46.308604 10924 raft_consensus.cc:359] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:46.309028 10924 raft_consensus.cc:385] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:46.309193 10924 raft_consensus.cc:740] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef94549942544e96b34eb069d6e3f8f6, State: Initialized, Role: FOLLOWER
I20260809 07:50:46.310029 10924 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
W20260809 07:50:46.311560 10677 tablet.cc:2406] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:50:46.311685 10957 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.312031 10957 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:46.312889 10924 sys_catalog.cc:565] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:50:46.313098 10952 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:50:46.313297 10952 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:50:46.311183 10652 master_runner.cc:187] Error getting master registration for 127.26.37.61:44571: OK, instance_id { permanent_uuid: "0819118351f44d1ca385876a50b18a95" instance_seqno: 1786261845781019 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:50:46.317849 10652 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:50:46.327838 10830 master_runner.cc:187] Error getting master registration for 127.26.37.61:44571: OK, instance_id { permanent_uuid: "0819118351f44d1ca385876a50b18a95" instance_seqno: 1786261845781019 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:50:46.334584 10830 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:50:46.337580 10973 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:50:46.337709 10973 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 07:50:46.477669 10859 tablet.cc:2406] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:50:46.488775 10827 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } attempt: 5
I20260809 07:50:46.501713 10827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: Bootstrap starting.
I20260809 07:50:46.506886 10827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:46.508316 10827 log.cc:826] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:46.511754 10827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: No bootstrap required, opened a new log
I20260809 07:50:46.519819 10827 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:46.520169 10827 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:46.520293 10827 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0819118351f44d1ca385876a50b18a95, State: Initialized, Role: FOLLOWER
I20260809 07:50:46.521021 10827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:46.522149 10979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.522527 10979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:46.523486 10827 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:46.545121 10742 master_runner.cc:264] No leader master found from master 0819118351f44d1ca385876a50b18a95
I20260809 07:50:46.545275 10742 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:50:46.549952 10993 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:50:46.550101 10993 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:50:46.581460 10934 raft_consensus.cc:493] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:50:46.581784 10934 raft_consensus.cc:515] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:46.583776 10934 leader_election.cc:290] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0819118351f44d1ca385876a50b18a95 (127.26.37.61:44571), ef94549942544e96b34eb069d6e3f8f6 (127.26.37.60:37563)
I20260809 07:50:46.585040 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03c6db696c2944eead26873a78123531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef94549942544e96b34eb069d6e3f8f6" is_pre_election: true
I20260809 07:50:46.585605 10885 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03c6db696c2944eead26873a78123531 in term 0.
I20260809 07:50:46.586567 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03c6db696c2944eead26873a78123531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0819118351f44d1ca385876a50b18a95" is_pre_election: true
I20260809 07:50:46.587119 10800 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03c6db696c2944eead26873a78123531 in term 0.
I20260809 07:50:46.588240 10675 leader_election.cc:304] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c6db696c2944eead26873a78123531, 0819118351f44d1ca385876a50b18a95; no voters:
I20260809 07:50:46.589111 10934 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:50:46.589272 10934 raft_consensus.cc:493] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:50:46.589388 10934 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:46.592969 10934 raft_consensus.cc:515] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:46.594277 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03c6db696c2944eead26873a78123531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0819118351f44d1ca385876a50b18a95"
I20260809 07:50:46.594478 10800 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:46.598094 10800 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03c6db696c2944eead26873a78123531 in term 1.
I20260809 07:50:46.598642 10675 leader_election.cc:304] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c6db696c2944eead26873a78123531, 0819118351f44d1ca385876a50b18a95; no voters:
I20260809 07:50:46.601608 10934 leader_election.cc:290] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [CANDIDATE]: Term 1 election: Requested vote from peers 0819118351f44d1ca385876a50b18a95 (127.26.37.61:44571), ef94549942544e96b34eb069d6e3f8f6 (127.26.37.60:37563)
I20260809 07:50:46.602211 10934 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:50:46.602725 10934 raft_consensus.cc:697] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 1 LEADER]: Becoming Leader. State: Replica: 03c6db696c2944eead26873a78123531, State: Running, Role: LEADER
I20260809 07:50:46.603551 10934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:46.613569 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03c6db696c2944eead26873a78123531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef94549942544e96b34eb069d6e3f8f6"
I20260809 07:50:46.613807 10885 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:46.617623 10885 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03c6db696c2944eead26873a78123531 in term 1.
I20260809 07:50:46.623293 10998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03c6db696c2944eead26873a78123531. Latest consensus state: current_term: 1 leader_uuid: "03c6db696c2944eead26873a78123531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.623736 10998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:46.630832 11000 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:50:46.633731 11000 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:50:46.654928 10885 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 1 FOLLOWER]: Refusing update from remote peer 03c6db696c2944eead26873a78123531: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:50:46.656189 10998 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:50:46.664067 10800 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 1 FOLLOWER]: Refusing update from remote peer 03c6db696c2944eead26873a78123531: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:50:46.665107 10998 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:50:46.684656 10957 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03c6db696c2944eead26873a78123531. Latest consensus state: current_term: 1 leader_uuid: "03c6db696c2944eead26873a78123531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.684963 10957 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:46.689383 11001 mvcc.cc:204] Tried to move back new op lower bound from 7316528523884879872 to 7316528523700842496. Current Snapshot: MvccSnapshot[applied={T|T < 7316528523884879872}]
I20260809 07:50:46.691434 10979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03c6db696c2944eead26873a78123531. Latest consensus state: current_term: 1 leader_uuid: "03c6db696c2944eead26873a78123531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.691702 10979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:46.704464 11002 mvcc.cc:204] Tried to move back new op lower bound from 7316528523884879872 to 7316528523700842496. Current Snapshot: MvccSnapshot[applied={T|T < 7316528523884879872}]
I20260809 07:50:46.705935 10979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03c6db696c2944eead26873a78123531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.706161 10979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:46.725073 10957 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03c6db696c2944eead26873a78123531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.725313 10957 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:46.728462 11000 catalog_manager.cc:1383] Generated new cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:50:46.728588 11000 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:50:46.747277 10934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03c6db696c2944eead26873a78123531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.747605 10934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:46.748073 10934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03c6db696c2944eead26873a78123531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:46.748303 10934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:46.804098 11000 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:50:46.805907 11000 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:50:46.828958 11000 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Generated new TSK 0
I20260809 07:50:46.830008 11000 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:50:46.841068 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:50:47.210088 11011 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:47.210383 11011 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:47.210458 11011 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:47.218513 11011 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:47.218663 11011 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:47.228688 11011 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:47.230548 11011 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:47.232442 11011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:47.241957 11020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:47.241964 11021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:47.243010 11024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:47.243773 11011 server_base.cc:1061] running on GCE node
I20260809 07:50:47.244243 11011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:47.245379 11011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:47.246747 11011 hybrid_clock.cc:648] HybridClock initialized: now 1786261847246620 us; error 95 us; skew 500 ppm
I20260809 07:50:47.247068 11011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:47.260713 11011 webserver.cc:533] Webserver started at http://127.26.37.1:38347/ using document root <none> and password file <none>
I20260809 07:50:47.261653 11011 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:47.261808 11011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:47.262209 11011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:47.264796 11011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:47.265727 11011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:47.271334 11011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:50:47.274755 11031 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:47.276328 11011 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:50:47.276470 11011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:47.276647 11011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:47.292716 11011 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:47.293473 11011 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:47.293702 11011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:47.294613 11011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:47.296133 11011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:47.296216 11011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:47.296305 11011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:47.296363 11011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:47.331106 11180 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:42877 every 8 connection(s)
I20260809 07:50:47.331104 11011 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:42877
I20260809 07:50:47.332937 11011 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:47.340543 10973 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:50:47.340651 10973 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: loading cluster ID for follower catalog manager: success
I20260809 07:50:47.341394 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11011
I20260809 07:50:47.341611 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:50:47.345774 10973 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: acquiring CA information for follower catalog manager: success
I20260809 07:50:47.347311 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:47.355588 10973 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:50:47.388072 11181 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:50:47.388485 11181 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:47.388725 11183 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:50:47.388906 11183 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:47.390113 11181 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:50:47.390244 11183 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:50:47.391399 11182 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:50:47.391542 11182 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:47.392028 11182 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:50:47.392549 10693 ts_manager.cc:194] Registered new tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:47.392561 10784 ts_manager.cc:194] Registered new tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:47.393883 10873 ts_manager.cc:194] Registered new tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:47.394338 10693 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:33217
I20260809 07:50:47.552003 10993 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:50:47.552099 10993 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: loading cluster ID for follower catalog manager: success
I20260809 07:50:47.554708 10993 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: acquiring CA information for follower catalog manager: success
I20260809 07:50:47.556386 10993 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:50:47.734902 11193 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:47.735277 11193 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:47.735427 11193 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:47.743908 11193 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:47.744127 11193 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:50:47.754532 11193 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:47.756227 11193 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:47.758248 11193 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:47.768318 11200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:47.768193 11199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:47.769788 11202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:47.770462 11193 server_base.cc:1061] running on GCE node
I20260809 07:50:47.771180 11193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:47.772333 11193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:47.773622 11193 hybrid_clock.cc:648] HybridClock initialized: now 1786261847773466 us; error 128 us; skew 500 ppm
I20260809 07:50:47.773939 11193 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:47.775942 11193 webserver.cc:533] Webserver started at http://127.26.37.2:34661/ using document root <none> and password file <none>
I20260809 07:50:47.776674 11193 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:47.776810 11193 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:47.777187 11193 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:47.779708 11193 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "7091f4eb014648a6a050226884b51bdb"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:47.780510 11193 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "7091f4eb014648a6a050226884b51bdb"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:47.785848 11193 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 07:50:47.789285 11210 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:47.790952 11193 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:50:47.791118 11193 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "7091f4eb014648a6a050226884b51bdb"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:47.791340 11193 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:47.817440 11193 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:47.818231 11193 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:47.818485 11193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:47.819371 11193 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:47.821049 11193 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:47.821179 11193 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:47.821282 11193 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:47.821394 11193 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:47.857136 11193 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:33731
I20260809 07:50:47.857219 11358 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:33731 every 8 connection(s)
I20260809 07:50:47.858728 11193 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:50:47.864578 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11193
I20260809 07:50:47.864840 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:50:47.869917 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:47.877950 11359 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:50:47.878065 11360 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:50:47.878336 11359 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:47.878343 11360 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:47.879340 11359 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:50:47.879396 11360 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:50:47.880347 11361 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:50:47.880518 11361 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:47.880996 11361 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:50:47.881058 10693 ts_manager.cc:194] Registered new tserver with Master: 7091f4eb014648a6a050226884b51bdb (127.26.37.2:33731)
I20260809 07:50:47.881273 10873 ts_manager.cc:194] Registered new tserver with Master: 7091f4eb014648a6a050226884b51bdb (127.26.37.2:33731)
I20260809 07:50:47.882090 10693 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:39817
I20260809 07:50:47.883105 10784 ts_manager.cc:194] Registered new tserver with Master: 7091f4eb014648a6a050226884b51bdb (127.26.37.2:33731)
W20260809 07:50:48.233348 11366 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:48.233753 11366 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:48.233914 11366 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:48.242853 11366 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:48.243093 11366 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:50:48.253501 11366 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:48.255234 11366 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:48.257068 11366 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:48.267807 11373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:48.267918 11376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:48.268268 11372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:48.269425 11366 server_base.cc:1061] running on GCE node
I20260809 07:50:48.270033 11366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:48.271165 11366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:48.272428 11366 hybrid_clock.cc:648] HybridClock initialized: now 1786261848272248 us; error 143 us; skew 500 ppm
I20260809 07:50:48.272812 11366 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:48.275204 11366 webserver.cc:533] Webserver started at http://127.26.37.3:35067/ using document root <none> and password file <none>
I20260809 07:50:48.276134 11366 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:48.276278 11366 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:48.276661 11366 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:50:48.279134 11366 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "bc562f8024ab425393a00b358cde1548"
format_stamp: "Formatted at 2026-08-09 07:50:48 on dist-test-slave-bw88"
I20260809 07:50:48.279923 11366 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "bc562f8024ab425393a00b358cde1548"
format_stamp: "Formatted at 2026-08-09 07:50:48 on dist-test-slave-bw88"
I20260809 07:50:48.285189 11366 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:50:48.288403 11383 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:48.289948 11366 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:48.290128 11366 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "bc562f8024ab425393a00b358cde1548"
format_stamp: "Formatted at 2026-08-09 07:50:48 on dist-test-slave-bw88"
I20260809 07:50:48.290343 11366 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:48.335042 11366 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:48.336050 11366 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:48.336357 11366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:48.337478 11366 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:48.339545 11366 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:48.339660 11366 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:48.339795 11366 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:48.339859 11366 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:48.377875 11366 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:39185
I20260809 07:50:48.377969 11541 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:39185 every 8 connection(s)
I20260809 07:50:48.379411 11366 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:50:48.379774 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11366
I20260809 07:50:48.379956 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:50:48.396869 11181 heartbeater.cc:499] Master 127.26.37.62:39655 was elected leader, sending a full tablet report...
I20260809 07:50:48.402446 11543 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:50:48.402894 11543 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:48.404001 11543 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:50:48.405072 11547 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:50:48.405241 11547 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:48.405354 11546 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:50:48.405515 11546 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:48.405737 11547 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:50:48.405860 10693 ts_manager.cc:194] Registered new tserver with Master: bc562f8024ab425393a00b358cde1548 (127.26.37.3:39185)
I20260809 07:50:48.406083 11546 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:50:48.406716 10784 ts_manager.cc:194] Registered new tserver with Master: bc562f8024ab425393a00b358cde1548 (127.26.37.3:39185)
I20260809 07:50:48.406827 10693 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:45435
I20260809 07:50:48.407416 10873 ts_manager.cc:194] Registered new tserver with Master: bc562f8024ab425393a00b358cde1548 (127.26.37.3:39185)
I20260809 07:50:48.410760 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:50:48.418072 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:50:48.418298 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 10652
I20260809 07:50:48.419353 10730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:48.529788 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 10652
I20260809 07:50:48.558977 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39655
--webserver_interface=127.26.37.62
--webserver_port=42013
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563 with env {}
I20260809 07:50:48.884404 11359 heartbeater.cc:499] Master 127.26.37.62:39655 was elected leader, sending a full tablet report...
W20260809 07:50:48.885984 11359 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:39655 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:39655: connect: Connection refused (error 111)
W20260809 07:50:48.906129 11558 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:48.906529 11558 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:48.906672 11558 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:48.914945 11558 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:48.915019 11558 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:48.915112 11558 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:48.915186 11558 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:48.925191 11558 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=42013
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:48.926779 11558 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:48.928580 11558 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:48.936386 11565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:48.936411 11567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:48.937736 11570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:48.938014 11558 server_base.cc:1061] running on GCE node
I20260809 07:50:48.938776 11558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:48.940271 11558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:48.941486 11558 hybrid_clock.cc:648] HybridClock initialized: now 1786261848941351 us; error 104 us; skew 500 ppm
I20260809 07:50:48.941915 11558 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:48.944031 11558 webserver.cc:533] Webserver started at http://127.26.37.62:42013/ using document root <none> and password file <none>
I20260809 07:50:48.944875 11558 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:48.945017 11558 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:48.950356 11558 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:50:48.953146 11577 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:48.954665 11558 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:50:48.954842 11558 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "03c6db696c2944eead26873a78123531"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:50:48.955528 11558 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:48.968772 11558 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:48.969619 11558 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:48.969909 11558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:48.990125 11558 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39655
I20260809 07:50:48.990144 11650 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39655 every 8 connection(s)
I20260809 07:50:48.991827 11558 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:48.993036 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11558
I20260809 07:50:48.993419 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11011
I20260809 07:50:49.001147 11651 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:50:49.003623 11172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:49.005172 11651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap starting.
I20260809 07:50:49.023108 11651 log.cc:826] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:49.032153 11651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:50:49.032698 11651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap complete.
I20260809 07:50:49.039508 11651 raft_consensus.cc:359] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:49.040226 11651 raft_consensus.cc:740] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c6db696c2944eead26873a78123531, State: Initialized, Role: FOLLOWER
I20260809 07:50:49.040956 11651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:49.041915 11661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:49.042219 11661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:49.043005 11651 sys_catalog.cc:565] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:49.081028 11675 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:50:49.081202 11675 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: loading cluster ID for follower catalog manager: success
I20260809 07:50:49.085126 11675 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: acquiring CA information for follower catalog manager: success
I20260809 07:50:49.087807 11675 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:50:49.117403 11558 master_runner.cc:264] No leader master found from master 03c6db696c2944eead26873a78123531
I20260809 07:50:49.117597 11558 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:50:49.120674 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11011
I20260809 07:50:49.146380 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:42877
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--webserver_port=38347
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:49.409490 11543 heartbeater.cc:499] Master 127.26.37.62:39655 was elected leader, sending a full tablet report...
I20260809 07:50:49.421770 11605 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "bc562f8024ab425393a00b358cde1548" instance_seqno: 1786261848369412) as {username='slave'} at 127.26.37.3:52541; Asking this server to re-register.
I20260809 07:50:49.422631 11543 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:49.424415 11605 ts_manager.cc:194] Registered new tserver with Master: bc562f8024ab425393a00b358cde1548 (127.26.37.3:39185)
W20260809 07:50:49.528342 11684 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:49.528708 11684 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:49.528846 11684 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:49.536796 11684 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:49.536993 11684 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:49.546952 11684 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:42877
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38347
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:49.548659 11684 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:49.550515 11684 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:49.560130 11696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:49.560571 11695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:49.560629 11700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:49.561992 11684 server_base.cc:1061] running on GCE node
I20260809 07:50:49.562584 11684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:49.563659 11684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:49.564865 11684 hybrid_clock.cc:648] HybridClock initialized: now 1786261849564733 us; error 104 us; skew 500 ppm
I20260809 07:50:49.565186 11684 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:49.567401 11684 webserver.cc:533] Webserver started at http://127.26.37.1:38347/ using document root <none> and password file <none>
I20260809 07:50:49.568164 11684 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:49.568305 11684 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:49.573714 11684 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:49.576478 11707 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:49.577920 11684 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:49.578100 11684 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:49.578794 11684 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:49.616390 11684 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:49.617254 11684 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:49.617583 11684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:49.618541 11684 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:49.620355 11684 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:49.620471 11684 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:49.620620 11684 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:49.620692 11684 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:49.656878 11684 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:42877
I20260809 07:50:49.656971 11857 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:42877 every 8 connection(s)
I20260809 07:50:49.658520 11684 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:49.667529 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11684
I20260809 07:50:49.685858 11858 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:50:49.686287 11858 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:49.687439 11858 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:50:49.689567 11605 ts_manager.cc:194] Registered new tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:49.690716 11859 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:50:49.690867 11859 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:49.691354 11859 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:50:49.692771 10873 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:49.696113 11860 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:50:49.696264 11860 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:49.696750 11860 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:50:49.698094 10784 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:49.703117 11872 raft_consensus.cc:493] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 03c6db696c2944eead26873a78123531)
I20260809 07:50:49.703444 11872 raft_consensus.cc:515] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:49.712275 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef94549942544e96b34eb069d6e3f8f6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0819118351f44d1ca385876a50b18a95" is_pre_election: true
I20260809 07:50:49.713132 11872 leader_election.cc:290] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03c6db696c2944eead26873a78123531 (127.26.37.62:39655), 0819118351f44d1ca385876a50b18a95 (127.26.37.61:44571)
I20260809 07:50:49.718015 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11684
I20260809 07:50:49.726953 11618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef94549942544e96b34eb069d6e3f8f6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "03c6db696c2944eead26873a78123531" is_pre_election: true
I20260809 07:50:49.727659 11618 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef94549942544e96b34eb069d6e3f8f6 in term 1.
I20260809 07:50:49.728358 10852 leader_election.cc:304] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03c6db696c2944eead26873a78123531, ef94549942544e96b34eb069d6e3f8f6; no voters: 0819118351f44d1ca385876a50b18a95
I20260809 07:50:49.728902 11872 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:50:49.729049 11872 raft_consensus.cc:493] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 03c6db696c2944eead26873a78123531)
I20260809 07:50:49.729179 11872 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:50:49.731513 11872 raft_consensus.cc:515] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:49.732084 11872 leader_election.cc:290] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [CANDIDATE]: Term 2 election: Requested vote from peers 03c6db696c2944eead26873a78123531 (127.26.37.62:39655), 0819118351f44d1ca385876a50b18a95 (127.26.37.61:44571)
I20260809 07:50:49.732579 11618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef94549942544e96b34eb069d6e3f8f6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "03c6db696c2944eead26873a78123531"
I20260809 07:50:49.732630 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef94549942544e96b34eb069d6e3f8f6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0819118351f44d1ca385876a50b18a95"
I20260809 07:50:49.732760 11618 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:50:49.736564 11618 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef94549942544e96b34eb069d6e3f8f6 in term 2.
I20260809 07:50:49.737001 10852 leader_election.cc:304] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03c6db696c2944eead26873a78123531, ef94549942544e96b34eb069d6e3f8f6; no voters: 0819118351f44d1ca385876a50b18a95
I20260809 07:50:49.737468 11872 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:50:49.737929 11872 raft_consensus.cc:697] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 2 LEADER]: Becoming Leader. State: Replica: ef94549942544e96b34eb069d6e3f8f6, State: Running, Role: LEADER
I20260809 07:50:49.738480 11872 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:49.740986 11853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:49.741433 11877 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef94549942544e96b34eb069d6e3f8f6. Latest consensus state: current_term: 2 leader_uuid: "ef94549942544e96b34eb069d6e3f8f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:49.741866 11877 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:49.742660 11881 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:50:49.745241 11881 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:50:49.746345 11881 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:50:49.746430 11881 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:50:49.747845 11881 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:50:49.748958 11881 catalog_manager.cc:6103] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Loaded TSK: 0
I20260809 07:50:49.750154 11881 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:50:49.830942 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11684
I20260809 07:50:49.854955 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:42877
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--webserver_port=38347
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260809 07:50:49.893712 11359 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:50:49.893932 11359 heartbeater.cc:499] Master 127.26.37.62:39655 was elected leader, sending a full tablet report...
I20260809 07:50:49.894961 11605 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7091f4eb014648a6a050226884b51bdb" instance_seqno: 1786261847848574) as {username='slave'} at 127.26.37.2:41433; Asking this server to re-register.
I20260809 07:50:49.895553 11359 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:49.896694 11605 ts_manager.cc:194] Registered new tserver with Master: 7091f4eb014648a6a050226884b51bdb (127.26.37.2:33731)
I20260809 07:50:49.921013 11884 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 03c6db696c2944eead26873a78123531)
I20260809 07:50:49.921293 11884 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:50:49.922700 11884 leader_election.cc:290] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03c6db696c2944eead26873a78123531 (127.26.37.62:39655), ef94549942544e96b34eb069d6e3f8f6 (127.26.37.60:37563)
I20260809 07:50:49.923682 10884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0819118351f44d1ca385876a50b18a95" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ef94549942544e96b34eb069d6e3f8f6" is_pre_election: true
I20260809 07:50:49.930078 11618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0819118351f44d1ca385876a50b18a95" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "03c6db696c2944eead26873a78123531" is_pre_election: true
I20260809 07:50:49.930410 11618 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0819118351f44d1ca385876a50b18a95 in current term 2: Already voted for candidate ef94549942544e96b34eb069d6e3f8f6 in this term.
I20260809 07:50:49.931126 10762 leader_election.cc:304] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0819118351f44d1ca385876a50b18a95; no voters: 03c6db696c2944eead26873a78123531, ef94549942544e96b34eb069d6e3f8f6
I20260809 07:50:49.931528 11884 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:50:49.934711 11884 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 07:50:50.206523 11882 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:50.206822 11882 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:50.206939 11882 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:50:50.207065 11882 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:50.215238 11882 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:50.215458 11882 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:50.217717 10800 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Refusing update from remote peer ef94549942544e96b34eb069d6e3f8f6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:50:50.218494 11877 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:50:50.220391 11884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef94549942544e96b34eb069d6e3f8f6. Latest consensus state: current_term: 2 leader_uuid: "ef94549942544e96b34eb069d6e3f8f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:50.220662 11884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:50.222680 11872 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ef94549942544e96b34eb069d6e3f8f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:50.222954 11872 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:50.223055 11884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ef94549942544e96b34eb069d6e3f8f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:50.223274 11884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:50.224747 11618 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 2 FOLLOWER]: Refusing update from remote peer ef94549942544e96b34eb069d6e3f8f6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:50:50.225455 11872 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:50:50.229039 11897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ef94549942544e96b34eb069d6e3f8f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:50.229219 11882 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:42877
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38347
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:50.229375 11897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:50.230006 11893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef94549942544e96b34eb069d6e3f8f6. Latest consensus state: current_term: 2 leader_uuid: "ef94549942544e96b34eb069d6e3f8f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:50.230340 11893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:50:50.231355 11882 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:50.233173 11882 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:50.243040 11900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:50.243257 11904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:50.243257 11899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:50.243863 11882 server_base.cc:1061] running on GCE node
I20260809 07:50:50.244495 11882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:50.245746 11882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:50.249089 11882 hybrid_clock.cc:648] HybridClock initialized: now 1786261850248883 us; error 147 us; skew 500 ppm
I20260809 07:50:50.249446 11882 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:50.251694 11882 webserver.cc:533] Webserver started at http://127.26.37.1:38347/ using document root <none> and password file <none>
I20260809 07:50:50.252475 11882 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:50.252629 11882 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:50.258203 11882 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:50:50.260957 11910 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:50.262383 11882 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:50.262579 11882 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:50.263415 11882 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:50.291257 11882 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:50.292140 11882 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:50.292408 11882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:50.293452 11882 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:50.295248 11882 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:50.295377 11882 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:50.295504 11882 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:50.295593 11882 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:50:50.296007 11882 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:50:50.296116 11882 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:50:50.296203 11882 fault_injection.cc:59] injecting fault for kudu (pid 11882): FLAGS_fault_before_start (process will exit)
I20260809 07:50:50.322584 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:42877
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--webserver_port=38347
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:50:50.662148 11872 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ef94549942544e96b34eb069d6e3f8f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:50:50.662436 11872 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: LEADER
W20260809 07:50:50.679930 11921 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:50.680235 11921 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:50.680311 11921 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:50.688406 11921 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:50.688565 11921 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:50:50.698364 11921 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45031,127.26.37.21:41269,127.26.37.22:41077,127.26.37.23:45371,127.26.37.24:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:42877
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38347
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:50:50.699957 11921 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:50.701825 11921 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:50:50.711714 11933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:50.711771 11929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:50.712742 11935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:50.712807 11921 server_base.cc:1061] running on GCE node
I20260809 07:50:50.713480 11921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:50.714735 11921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:50.715976 11921 hybrid_clock.cc:648] HybridClock initialized: now 1786261850715821 us; error 121 us; skew 500 ppm
I20260809 07:50:50.716316 11921 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:50:50.718585 11921 webserver.cc:533] Webserver started at http://127.26.37.1:38347/ using document root <none> and password file <none>
I20260809 07:50:50.719440 11921 fs_manager.cc:362] Metadata directory not provided
I20260809 07:50:50.719581 11921 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:50.725661 11921 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:50:50.728236 11943 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:50.729796 11921 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:50.730019 11921 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:50:50.730688 11921 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:50.759390 11921 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:50:50.760290 11921 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:50.760612 11921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:50.761693 11921 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:50:50.763429 11921 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:50.763511 11921 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:50.763623 11921 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:50:50.763669 11921 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:50.803023 11921 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:42877
I20260809 07:50:50.803103 12109 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:42877 every 8 connection(s)
I20260809 07:50:50.804569 11921 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:50.809741 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11921
I20260809 07:50:50.810091 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11921
I20260809 07:50:50.827976 12110 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:50:50.828418 12110 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:50.829677 12110 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:50:50.830862 12112 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:50:50.831012 12112 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:50.831563 12112 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:50:50.832034 11605 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:50.832818 10784 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:50.835391 12111 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:50:50.835587 12111 heartbeater.cc:461] Registering TS with master...
I20260809 07:50:50.836095 12111 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:50:50.837131 10873 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:42877)
I20260809 07:50:50.837764 12104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:50.838964 10873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:48389
I20260809 07:50:50.887725 11360 heartbeater.cc:499] Master 127.26.37.60:37563 was elected leader, sending a full tablet report...
I20260809 07:50:50.926810 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11921
I20260809 07:50:50.955229 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11193
I20260809 07:50:50.963595 11352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:51.048487 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11193
I20260809 07:50:51.072889 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11366
I20260809 07:50:51.081722 11536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:51.175841 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11366
I20260809 07:50:51.200176 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11558
I20260809 07:50:51.206703 11646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:51.316435 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11558
W20260809 07:50:51.342682 10852 connection.cc:573] client connection to 127.26.37.62:39655 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:50:51.343014 10830 master_runner.cc:187] Error getting master registration for 127.26.37.62:39655: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260809 07:50:51.343158 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 10742
I20260809 07:50:51.343209 10830 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:50:51.344367 10821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:51.451634 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 10742
I20260809 07:50:51.477768 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 10830
I20260809 07:50:51.478958 10917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:50:51.588676 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 10830
Shutting down hive metastore.
2026-08-09T07:50:51Z chronyd exiting
2026-08-09T07:50:52Z chronyd exiting
2026-08-09T07:50:52Z chronyd exiting
2026-08-09T07:50:52Z chronyd exiting
2026-08-09T07:50:52Z chronyd exiting
2026-08-09T07:50:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:52Z Disabled control of system clock
2026-08-09T07:50:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:52Z Disabled control of system clock
2026-08-09T07:50:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:52Z Disabled control of system clock
2026-08-09T07:50:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:52Z Disabled control of system clock
2026-08-09T07:50:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:52Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:50:53: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:51:01.769081 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39655
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563 with env {}
W20260809 07:51:02.125901 12403 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:02.126475 12403 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:02.126622 12403 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:02.140864 12403 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:02.140991 12403 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:02.141091 12403 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:02.141186 12403 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:02.157035 12403 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:02.159512 12403 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:02.162339 12403 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:02.173206 12409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:02.173367 12408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:02.174507 12413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:02.175670 12403 server_base.cc:1061] running on GCE node
I20260809 07:51:02.176867 12403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:02.178727 12403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:02.180455 12403 hybrid_clock.cc:648] HybridClock initialized: now 1786261862180266 us; error 118 us; skew 500 ppm
I20260809 07:51:02.181039 12403 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:02.183979 12403 webserver.cc:533] Webserver started at http://127.26.37.62:36861/ using document root <none> and password file <none>
I20260809 07:51:02.185017 12403 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:02.185211 12403 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:02.190974 12403 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:51:02.193823 12422 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:02.195048 12403 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:51:02.195228 12403 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "03c6db696c2944eead26873a78123531"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:51:02.195928 12403 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:02.207794 12403 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:02.208575 12403 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:02.208878 12403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:02.229583 12403 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39655
I20260809 07:51:02.229614 12496 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39655 every 8 connection(s)
I20260809 07:51:02.231496 12403 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:02.235535 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12403
I20260809 07:51:02.235728 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:51:02.236960 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:44571
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563 with env {}
I20260809 07:51:02.240252 12497 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:02.245446 12497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap starting.
I20260809 07:51:02.263145 12497 log.cc:826] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:02.272262 12497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:02.272809 12497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap complete.
I20260809 07:51:02.279888 12497 raft_consensus.cc:359] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:02.280578 12497 raft_consensus.cc:740] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c6db696c2944eead26873a78123531, State: Initialized, Role: FOLLOWER
I20260809 07:51:02.281297 12497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:02.282235 12504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:02.282537 12504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:02.283156 12497 sys_catalog.cc:565] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:02.294072 12520 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:51:02.294265 12520 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: loading cluster ID for follower catalog manager: success
I20260809 07:51:02.299424 12520 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: acquiring CA information for follower catalog manager: success
I20260809 07:51:02.302605 12520 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:02.315281 12403 master_runner.cc:187] Error getting master registration for 127.26.37.61:44571: Network error: Client connection negotiation failed: client connection to 127.26.37.61:44571: connect: Connection refused (error 111),
I20260809 07:51:02.319195 12403 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:51:02.621976 12501 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:02.622357 12501 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:02.622490 12501 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:02.630858 12501 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:02.630998 12501 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:02.631119 12501 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:02.631203 12501 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:02.641328 12501 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:02.643139 12501 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:02.645066 12501 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:02.653731 12533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:02.653740 12532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:02.655171 12536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:02.656201 12501 server_base.cc:1061] running on GCE node
I20260809 07:51:02.657043 12501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:02.658885 12501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:02.660126 12501 hybrid_clock.cc:648] HybridClock initialized: now 1786261862659977 us; error 121 us; skew 500 ppm
I20260809 07:51:02.660501 12501 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:02.662894 12501 webserver.cc:533] Webserver started at http://127.26.37.61:33051/ using document root <none> and password file <none>
I20260809 07:51:02.663677 12501 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:02.663822 12501 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:02.669739 12501 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:51:02.672806 12546 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:02.674369 12501 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:02.674561 12501 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "0819118351f44d1ca385876a50b18a95"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:51:02.675338 12501 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:02.694362 12501 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:02.695291 12501 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:02.695611 12501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:02.716837 12501 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:44571
I20260809 07:51:02.716857 12624 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:44571 every 8 connection(s)
I20260809 07:51:02.718616 12501 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:51:02.724938 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12501
I20260809 07:51:02.725189 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:51:02.725821 12627 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:02.726326 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37563
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563 with env {}
I20260809 07:51:02.729142 12627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: Bootstrap starting.
I20260809 07:51:02.745859 12627 log.cc:826] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:02.755005 12627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:02.755589 12627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: Bootstrap complete.
I20260809 07:51:02.762509 12627 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:02.763234 12627 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0819118351f44d1ca385876a50b18a95, State: Initialized, Role: FOLLOWER
I20260809 07:51:02.764040 12627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:02.764981 12634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:02.765286 12634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:02.766065 12627 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:02.780689 12648 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:51:02.780880 12648 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: loading cluster ID for follower catalog manager: success
I20260809 07:51:02.785473 12648 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: acquiring CA information for follower catalog manager: success
I20260809 07:51:02.788136 12648 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:02.804844 12501 master_runner.cc:187] Error getting master registration for 127.26.37.60:37563: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37563: connect: Connection refused (error 111),
I20260809 07:51:02.810747 12501 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:51:03.101002 12631 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:03.101429 12631 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:03.101583 12631 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:03.109822 12631 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:03.109946 12631 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:03.110080 12631 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:03.110162 12631 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:03.120122 12631 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:03.121804 12631 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:03.123663 12631 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:03.131688 12660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:03.132148 12661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:03.132907 12665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:03.134068 12631 server_base.cc:1061] running on GCE node
I20260809 07:51:03.134891 12631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:03.136458 12631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:03.137871 12631 hybrid_clock.cc:648] HybridClock initialized: now 1786261863137729 us; error 105 us; skew 500 ppm
I20260809 07:51:03.138258 12631 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:03.140555 12631 webserver.cc:533] Webserver started at http://127.26.37.60:35837/ using document root <none> and password file <none>
I20260809 07:51:03.141307 12631 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:03.141525 12631 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:03.147069 12631 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:51:03.150096 12671 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:03.151579 12631 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:03.151753 12631 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "ef94549942544e96b34eb069d6e3f8f6"
format_stamp: "Formatted at 2026-08-09 07:50:46 on dist-test-slave-bw88"
I20260809 07:51:03.152549 12631 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:03.178498 12631 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:03.179400 12631 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:03.179704 12631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:03.202100 12631 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:37563
I20260809 07:51:03.202136 12743 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:37563 every 8 connection(s)
I20260809 07:51:03.203778 12631 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:51:03.210871 12744 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:03.213658 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12631
I20260809 07:51:03.213847 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:51:03.214318 12744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Bootstrap starting.
I20260809 07:51:03.235185 12744 log.cc:826] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:03.246411 12744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:03.247007 12744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: Bootstrap complete.
I20260809 07:51:03.253489 12744 raft_consensus.cc:359] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:03.254245 12744 raft_consensus.cc:740] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef94549942544e96b34eb069d6e3f8f6, State: Initialized, Role: FOLLOWER
I20260809 07:51:03.255039 12744 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:03.256107 12749 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:03.256501 12749 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:03.257133 12744 sys_catalog.cc:565] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:03.270942 12771 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:51:03.271162 12771 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: loading cluster ID for follower catalog manager: success
I20260809 07:51:03.275727 12771 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: acquiring CA information for follower catalog manager: success
I20260809 07:51:03.279630 12771 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:03.294104 12631 master_runner.cc:264] No leader master found from master ef94549942544e96b34eb069d6e3f8f6
I20260809 07:51:03.294293 12631 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:04.030856 12774 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:04.031162 12774 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:04.033074 12774 leader_election.cc:290] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 03c6db696c2944eead26873a78123531 (127.26.37.62:39655), ef94549942544e96b34eb069d6e3f8f6 (127.26.37.60:37563)
I20260809 07:51:04.033953 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0819118351f44d1ca385876a50b18a95" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "03c6db696c2944eead26873a78123531" is_pre_election: true
I20260809 07:51:04.034461 12464 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0819118351f44d1ca385876a50b18a95 in term 2.
I20260809 07:51:04.035198 12547 leader_election.cc:304] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c6db696c2944eead26873a78123531, 0819118351f44d1ca385876a50b18a95; no voters:
I20260809 07:51:04.035606 12774 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:51:04.035769 12774 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:04.035866 12774 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:04.040597 12774 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:04.040845 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0819118351f44d1ca385876a50b18a95" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ef94549942544e96b34eb069d6e3f8f6" is_pre_election: true
I20260809 07:51:04.041306 12774 leader_election.cc:290] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [CANDIDATE]: Term 3 election: Requested vote from peers 03c6db696c2944eead26873a78123531 (127.26.37.62:39655), ef94549942544e96b34eb069d6e3f8f6 (127.26.37.60:37563)
I20260809 07:51:04.041381 12707 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0819118351f44d1ca385876a50b18a95 in term 2.
I20260809 07:51:04.041762 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0819118351f44d1ca385876a50b18a95" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "03c6db696c2944eead26873a78123531"
I20260809 07:51:04.041915 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0819118351f44d1ca385876a50b18a95" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ef94549942544e96b34eb069d6e3f8f6"
I20260809 07:51:04.041985 12464 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:04.042157 12706 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:04.047284 12706 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0819118351f44d1ca385876a50b18a95 in term 3.
I20260809 07:51:04.047277 12464 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0819118351f44d1ca385876a50b18a95 in term 3.
I20260809 07:51:04.047741 12547 leader_election.cc:304] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c6db696c2944eead26873a78123531, 0819118351f44d1ca385876a50b18a95; no voters:
I20260809 07:51:04.048059 12774 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:51:04.048399 12774 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [term 3 LEADER]: Becoming Leader. State: Replica: 0819118351f44d1ca385876a50b18a95, State: Running, Role: LEADER
I20260809 07:51:04.048872 12774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:04.051788 12778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0819118351f44d1ca385876a50b18a95. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:04.052075 12778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:04.052702 12782 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:51:04.055002 12782 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:51:04.055804 12782 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:51:04.055902 12782 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:51:04.057030 12782 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:51:04.057874 12782 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95: Loaded TSK: 0
I20260809 07:51:04.058806 12782 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:51:04.063024 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:51:04.432220 12786 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:04.432512 12786 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:04.432591 12786 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:04.440659 12786 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:04.440814 12786 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:04.450594 12786 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:04.452601 12786 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:04.454504 12786 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:04.464099 12792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:04.464143 12793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:04.464999 12795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:04.465534 12786 server_base.cc:1061] running on GCE node
I20260809 07:51:04.466174 12786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:04.467335 12786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:04.468597 12786 hybrid_clock.cc:648] HybridClock initialized: now 1786261864468459 us; error 108 us; skew 500 ppm
I20260809 07:51:04.468931 12786 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:04.471297 12786 webserver.cc:533] Webserver started at http://127.26.37.1:42609/ using document root <none> and password file <none>
I20260809 07:51:04.472090 12786 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:04.472229 12786 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:04.477654 12786 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:04.480681 12804 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:04.481980 12706 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [term 3 FOLLOWER]: Refusing update from remote peer 0819118351f44d1ca385876a50b18a95: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:51:04.482329 12786 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:04.483007 12778 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:51:04.482502 12786 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:51:04.483388 12786 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:04.487155 12805 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0819118351f44d1ca385876a50b18a95. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:04.487434 12805 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:04.494238 12805 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:04.494491 12805 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef94549942544e96b34eb069d6e3f8f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:04.496414 12819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:04.496722 12819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:04.496712 12464 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 3 FOLLOWER]: Refusing update from remote peer 0819118351f44d1ca385876a50b18a95: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:51:04.497531 12819 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:51:04.501647 12826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:04.502038 12826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:04.502039 12820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0819118351f44d1ca385876a50b18a95. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:04.502372 12820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:04.528124 12786 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:04.528955 12786 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:04.529227 12786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:04.530344 12786 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:04.532038 12786 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:04.532123 12786 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:04.532203 12786 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:04.532248 12786 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:04.568948 12786 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:37855
I20260809 07:51:04.569018 12991 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:37855 every 8 connection(s)
I20260809 07:51:04.570706 12786 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:04.571120 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12786
I20260809 07:51:04.571388 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:51:04.578821 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:51:04.607690 12996 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:51:04.607750 12992 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:51:04.608173 12996 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:04.608186 12992 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:04.609210 12996 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:51:04.609241 12992 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:51:04.611639 12450 ts_manager.cc:194] Registered new tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:37855)
I20260809 07:51:04.611961 12696 ts_manager.cc:194] Registered new tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:37855)
I20260809 07:51:04.617897 12998 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:51:04.618054 12998 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:04.618458 12998 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:51:04.619798 12569 ts_manager.cc:194] Registered new tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:37855)
I20260809 07:51:04.621541 12569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:46991
W20260809 07:51:04.936342 13007 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:04.936630 13007 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:04.936699 13007 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:04.944797 13007 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:04.944947 13007 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:51:04.954735 13007 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:04.956353 13007 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:04.958312 13007 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:04.967573 13015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:04.967612 13014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:04.968781 13017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:04.969718 13007 server_base.cc:1061] running on GCE node
I20260809 07:51:04.970387 13007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:04.971601 13007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:04.972833 13007 hybrid_clock.cc:648] HybridClock initialized: now 1786261864972694 us; error 109 us; skew 500 ppm
I20260809 07:51:04.973155 13007 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:04.975293 13007 webserver.cc:533] Webserver started at http://127.26.37.2:34689/ using document root <none> and password file <none>
I20260809 07:51:04.976058 13007 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:04.976213 13007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:04.981948 13007 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 07:51:04.984724 13023 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:04.986150 13007 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:04.986336 13007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "7091f4eb014648a6a050226884b51bdb"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:51:04.987210 13007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:05.011682 13007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:05.012521 13007 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:05.012827 13007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:05.013854 13007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:05.015511 13007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:05.015614 13007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.015758 13007 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:05.015839 13007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.050664 13007 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:41155
I20260809 07:51:05.050722 13201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:41155 every 8 connection(s)
I20260809 07:51:05.052186 13007 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:05.062645 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13007
I20260809 07:51:05.062840 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:51:05.068756 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:51:05.070232 13202 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:51:05.070364 13206 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:51:05.070647 13202 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:05.070652 13206 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:05.071578 13202 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:51:05.071621 13206 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:51:05.072618 13204 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:51:05.072763 13204 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:05.073257 13204 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:51:05.073653 12450 ts_manager.cc:194] Registered new tserver with Master: 7091f4eb014648a6a050226884b51bdb (127.26.37.2:41155)
I20260809 07:51:05.074136 12569 ts_manager.cc:194] Registered new tserver with Master: 7091f4eb014648a6a050226884b51bdb (127.26.37.2:41155)
I20260809 07:51:05.074496 12696 ts_manager.cc:194] Registered new tserver with Master: 7091f4eb014648a6a050226884b51bdb (127.26.37.2:41155)
I20260809 07:51:05.075145 12569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:53257
I20260809 07:51:05.089525 13213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:05.089942 13213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: LEADER
W20260809 07:51:05.420750 13212 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:05.421025 13212 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:05.421228 13212 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:05.430372 13212 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:05.430568 13212 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:51:05.440097 13212 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:05.441766 13212 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:05.443565 13212 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:05.453312 13223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:05.453293 13222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:05.453699 13226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:05.454267 13212 server_base.cc:1061] running on GCE node
I20260809 07:51:05.454921 13212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:05.456023 13212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:05.457253 13212 hybrid_clock.cc:648] HybridClock initialized: now 1786261865457111 us; error 114 us; skew 500 ppm
I20260809 07:51:05.457656 13212 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:05.459784 13212 webserver.cc:533] Webserver started at http://127.26.37.3:46635/ using document root <none> and password file <none>
I20260809 07:51:05.460546 13212 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:05.460704 13212 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:05.466001 13212 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:05.468811 13236 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.470256 13212 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:51:05.470432 13212 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "bc562f8024ab425393a00b358cde1548"
format_stamp: "Formatted at 2026-08-09 07:50:48 on dist-test-slave-bw88"
I20260809 07:51:05.471139 13212 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:05.498698 13212 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:05.499616 13212 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:05.499920 13212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:05.500942 13212 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:05.502876 13212 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:05.502950 13212 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.503036 13212 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:05.503087 13212 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.538179 13212 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:41825
I20260809 07:51:05.538228 13386 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:41825 every 8 connection(s)
I20260809 07:51:05.539855 13212 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:05.540153 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13212
I20260809 07:51:05.540387 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:51:05.557832 13387 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:51:05.557888 13388 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:51:05.557914 13390 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:51:05.558302 13387 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:05.558307 13388 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:05.558307 13390 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:05.559558 13390 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:51:05.559590 13388 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:51:05.559653 13387 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:51:05.561866 12696 ts_manager.cc:194] Registered new tserver with Master: bc562f8024ab425393a00b358cde1548 (127.26.37.3:41825)
I20260809 07:51:05.561903 12569 ts_manager.cc:194] Registered new tserver with Master: bc562f8024ab425393a00b358cde1548 (127.26.37.3:41825)
I20260809 07:51:05.562881 12450 ts_manager.cc:194] Registered new tserver with Master: bc562f8024ab425393a00b358cde1548 (127.26.37.3:41825)
I20260809 07:51:05.563017 12569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:55529
I20260809 07:51:05.568459 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:05.576226 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:05.576436 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12403
I20260809 07:51:05.577329 12491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:05.623929 12998 heartbeater.cc:499] Master 127.26.37.61:44571 was elected leader, sending a full tablet report...
I20260809 07:51:05.684592 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12403
W20260809 07:51:05.717770 12992 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:39655 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.26.37.62:39655 (error 108)
I20260809 07:51:05.718138 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39655
--webserver_interface=127.26.37.62
--webserver_port=36861
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563 with env {}
W20260809 07:51:05.980674 12547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:39655: connect: Connection refused (error 111)
W20260809 07:51:05.982352 12547 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 -> Peer 03c6db696c2944eead26873a78123531 (127.26.37.62:39655): Couldn't send request to peer 03c6db696c2944eead26873a78123531. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:39655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:51:06.057250 13399 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:06.057685 13399 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:06.057806 13399 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:06.066075 13399 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:06.066147 13399 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:06.066238 13399 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:06.066309 13399 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:06.077242 13206 heartbeater.cc:499] Master 127.26.37.61:44571 was elected leader, sending a full tablet report...
W20260809 07:51:06.077512 13202 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:39655 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:39655: connect: Connection refused (error 111)
I20260809 07:51:06.077519 13399 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=36861
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:06.079452 13399 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:06.081457 13399 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:06.089983 13412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.090016 13411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.090639 13417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:06.091203 13399 server_base.cc:1061] running on GCE node
I20260809 07:51:06.091993 13399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:06.093686 13399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:06.094918 13399 hybrid_clock.cc:648] HybridClock initialized: now 1786261866094763 us; error 127 us; skew 500 ppm
I20260809 07:51:06.095270 13399 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:06.097433 13399 webserver.cc:533] Webserver started at http://127.26.37.62:36861/ using document root <none> and password file <none>
I20260809 07:51:06.098152 13399 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:06.098295 13399 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:06.103543 13399 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 07:51:06.106359 13425 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.107617 13399 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:51:06.107801 13399 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "03c6db696c2944eead26873a78123531"
format_stamp: "Formatted at 2026-08-09 07:50:45 on dist-test-slave-bw88"
I20260809 07:51:06.108459 13399 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:06.127038 13399 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:06.127887 13399 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:06.128211 13399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:06.149384 13399 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39655
I20260809 07:51:06.149425 13501 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39655 every 8 connection(s)
I20260809 07:51:06.151244 13399 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:06.153234 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13399
I20260809 07:51:06.153580 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12786
I20260809 07:51:06.159794 13503 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:06.164381 13503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap starting.
I20260809 07:51:06.166271 12986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:06.188898 13503 log.cc:826] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:06.200389 13503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:06.201036 13503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: Bootstrap complete.
I20260809 07:51:06.210992 13503 raft_consensus.cc:359] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:06.211966 13503 raft_consensus.cc:740] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c6db696c2944eead26873a78123531, State: Initialized, Role: FOLLOWER
I20260809 07:51:06.212767 13503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } }
I20260809 07:51:06.213717 13512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:06.214112 13512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:06.214949 13503 sys_catalog.cc:565] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:06.225243 13526 catalog_manager.cc:1295] Loaded cluster ID: 19c476ce0e9a4b1bbe3aa609ab0744c9
I20260809 07:51:06.225415 13526 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: loading cluster ID for follower catalog manager: success
I20260809 07:51:06.229588 13526 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: acquiring CA information for follower catalog manager: success
I20260809 07:51:06.244887 13526 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:06.288828 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12786
I20260809 07:51:06.313242 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:37855
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--webserver_port=42609
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:51:06.565233 13390 heartbeater.cc:499] Master 127.26.37.61:44571 was elected leader, sending a full tablet report...
I20260809 07:51:06.577587 13452 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "bc562f8024ab425393a00b358cde1548" instance_seqno: 1786261865530130) as {username='slave'} at 127.26.37.3:48337; Asking this server to re-register.
I20260809 07:51:06.578473 13387 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:06.580416 13452 ts_manager.cc:194] Registered new tserver with Master: bc562f8024ab425393a00b358cde1548 (127.26.37.3:41825)
I20260809 07:51:06.602787 13213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:06.603065 13213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:06.616910 13512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0819118351f44d1ca385876a50b18a95. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:06.617235 13512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c6db696c2944eead26873a78123531 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:51:06.681951 13534 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:06.682312 13534 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:06.682456 13534 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:06.691139 13534 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:06.691354 13534 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:06.701275 13534 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:37855
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42609
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:06.702986 13534 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:06.704861 13534 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:06.714226 13549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.714195 13550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.715040 13552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:06.715438 13534 server_base.cc:1061] running on GCE node
I20260809 07:51:06.716040 13534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:06.717208 13534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:06.718547 13534 hybrid_clock.cc:648] HybridClock initialized: now 1786261866718369 us; error 124 us; skew 500 ppm
I20260809 07:51:06.718868 13534 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:06.721134 13534 webserver.cc:533] Webserver started at http://127.26.37.1:42609/ using document root <none> and password file <none>
I20260809 07:51:06.721956 13534 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:06.722100 13534 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:06.727551 13534 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:51:06.730273 13559 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.731633 13534 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:06.731765 13534 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:51:06.732448 13534 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:06.753701 13534 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:06.754594 13534 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:06.754868 13534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:06.755823 13534 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:06.757848 13534 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:06.757942 13534 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.758067 13534 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:06.758136 13534 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.794601 13534 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:37855
I20260809 07:51:06.794682 13711 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:37855 every 8 connection(s)
I20260809 07:51:06.796269 13534 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:06.801973 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13534
I20260809 07:51:06.830135 13712 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:51:06.830562 13712 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:06.831480 13712 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:51:06.833189 13452 ts_manager.cc:194] Registered new tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:37855)
I20260809 07:51:06.835919 13717 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:51:06.836091 13717 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:06.836597 13717 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:51:06.837779 12569 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:37855)
I20260809 07:51:06.838932 12569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:46935
I20260809 07:51:06.842856 13714 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:51:06.843029 13714 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:06.843431 13714 heartbeater.cc:507] Master 127.26.37.60:37563 requested a full tablet report, sending...
I20260809 07:51:06.844534 12696 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:37855)
I20260809 07:51:06.958935 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13534
I20260809 07:51:06.970088 13707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:07.056475 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13534
I20260809 07:51:07.086457 13202 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:51:07.086966 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:37855
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--webserver_port=42609
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260809 07:51:07.087884 13452 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7091f4eb014648a6a050226884b51bdb" instance_seqno: 1786261865042354) as {username='slave'} at 127.26.37.2:35085; Asking this server to re-register.
I20260809 07:51:07.088513 13202 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:07.089960 13452 ts_manager.cc:194] Registered new tserver with Master: 7091f4eb014648a6a050226884b51bdb (127.26.37.2:41155)
I20260809 07:51:07.224404 13537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0819118351f44d1ca385876a50b18a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c6db696c2944eead26873a78123531" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39655 } } peers { permanent_uuid: "0819118351f44d1ca385876a50b18a95" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 44571 } } peers { permanent_uuid: "ef94549942544e96b34eb069d6e3f8f6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37563 } } }
I20260809 07:51:07.224753 13537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0819118351f44d1ca385876a50b18a95 [sys.catalog]: This master's current role is: LEADER
W20260809 07:51:07.445312 13734 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:07.445657 13734 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:07.445741 13734 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:51:07.445798 13734 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:07.453847 13734 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:07.454021 13734 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:07.464250 13734 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:37855
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42609
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:07.465919 13734 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:07.467849 13734 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:07.478258 13743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:07.478709 13746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:07.478255 13744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:07.479858 13734 server_base.cc:1061] running on GCE node
I20260809 07:51:07.480481 13734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:07.481779 13734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:07.483036 13734 hybrid_clock.cc:648] HybridClock initialized: now 1786261867482823 us; error 138 us; skew 500 ppm
I20260809 07:51:07.483284 13734 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:07.485639 13734 webserver.cc:533] Webserver started at http://127.26.37.1:42609/ using document root <none> and password file <none>
I20260809 07:51:07.486382 13734 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:07.486490 13734 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:07.491981 13734 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:51:07.494947 13755 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.496301 13734 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:51:07.496448 13734 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:51:07.497134 13734 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:07.527158 13734 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:07.528119 13734 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:07.528368 13734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:07.529307 13734 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:07.531132 13734 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:07.531225 13734 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.531309 13734 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:07.531369 13734 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:51:07.531711 13734 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:51:07.531845 13734 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:51:07.531937 13734 fault_injection.cc:59] injecting fault for kudu (pid 13734): FLAGS_fault_before_start (process will exit)
I20260809 07:51:07.557008 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:37855
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--webserver_port=42609
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:51:07.912467 13766 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:07.912827 13766 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:07.912971 13766 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:07.921316 13766 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:07.921479 13766 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:07.930909 13766 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:46321,127.26.37.21:40307,127.26.37.22:34989,127.26.37.23:45299,127.26.37.24:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:37855
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42609
--tserver_master_addrs=127.26.37.62:39655,127.26.37.61:44571,127.26.37.60:37563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:07.932543 13766 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:07.934374 13766 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:07.943717 13773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:07.943773 13774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:07.944278 13776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:07.945127 13766 server_base.cc:1061] running on GCE node
I20260809 07:51:07.945734 13766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:07.946841 13766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:07.948048 13766 hybrid_clock.cc:648] HybridClock initialized: now 1786261867947917 us; error 101 us; skew 500 ppm
I20260809 07:51:07.948359 13766 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:07.950486 13766 webserver.cc:533] Webserver started at http://127.26.37.1:42609/ using document root <none> and password file <none>
I20260809 07:51:07.951232 13766 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:07.951359 13766 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:07.956842 13766 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:51:07.959734 13786 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.961174 13766 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 07:51:07.961403 13766 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bd1b9f933e744a1b8bb814a14b2e8a7d"
format_stamp: "Formatted at 2026-08-09 07:50:47 on dist-test-slave-bw88"
I20260809 07:51:07.962131 13766 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:07.986814 13766 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:07.987727 13766 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:07.988034 13766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:07.989035 13766 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:07.990839 13766 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:07.990947 13766 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.991117 13766 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:07.991191 13766 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:08.027247 13766 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:37855
I20260809 07:51:08.027339 13950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:37855 every 8 connection(s)
I20260809 07:51:08.029027 13766 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:08.034597 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13766
I20260809 07:51:08.034927 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13766
I20260809 07:51:08.049592 13951 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39655
I20260809 07:51:08.050020 13951 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:08.051180 13951 heartbeater.cc:507] Master 127.26.37.62:39655 requested a full tablet report, sending...
I20260809 07:51:08.055303 13452 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:37855)
I20260809 07:51:08.059087 13953 heartbeater.cc:344] Connected to a master server at 127.26.37.61:44571
I20260809 07:51:08.059257 13953 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:08.059849 13953 heartbeater.cc:507] Master 127.26.37.61:44571 requested a full tablet report, sending...
I20260809 07:51:08.060220 13945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:08.061146 12569 ts_manager.cc:194] Re-registered known tserver with Master: bd1b9f933e744a1b8bb814a14b2e8a7d (127.26.37.1:37855)
I20260809 07:51:08.062119 12569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:50489
I20260809 07:51:08.153060 13952 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37563
I20260809 07:51:08.153072 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13766
I20260809 07:51:08.176785 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13007
I20260809 07:51:08.185058 13191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:08.271627 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13007
I20260809 07:51:08.295275 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13212
I20260809 07:51:08.304698 13380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:08.392982 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13212
I20260809 07:51:08.417333 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13399
I20260809 07:51:08.424189 13496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:08.530609 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13399
I20260809 07:51:08.559469 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12501
I20260809 07:51:08.560751 12619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:08.670989 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12501
I20260809 07:51:08.699954 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12631
I20260809 07:51:08.701144 12737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:08.834297 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12631
Shutting down hive metastore.
2026-08-09T07:51:09Z chronyd exiting
2026-08-09T07:51:09Z chronyd exiting
2026-08-09T07:51:09Z chronyd exiting
2026-08-09T07:51:09Z chronyd exiting
2026-08-09T07:51:09Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/5 (40828 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/6
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:51:09 dist-test-slave-bw88 krb5kdc[13991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:09 dist-test-slave-bw88 krb5kdc[13991](info): set up 2 sockets
Aug 09 07:51:09 dist-test-slave-bw88 krb5kdc[13991](info): commencing operation
krb5kdc: starting...
W20260809 07:51:11.508162 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.184s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:51:11 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:11.792778 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40109
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:12.143139 14010 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:12.143514 14010 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:12.143697 14010 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:12.151847 14010 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:12.151947 14010 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:12.152036 14010 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:12.152112 14010 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:12.162653 14010 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:12.164508 14010 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:12.166390 14010 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:12.178056 14016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:12.178053 14017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:12.179134 14020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:12.179543 14010 server_base.cc:1061] running on GCE node
I20260809 07:51:12.180335 14010 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:12.180542 14010 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:12.180647 14010 hybrid_clock.cc:648] HybridClock initialized: now 1786261872180623 us; error 0 us; skew 500 ppm
I20260809 07:51:12.181010 14010 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:12 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:12.184521 14010 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:12.187110 14010 webserver.cc:533] Webserver started at http://127.26.37.62:33779/ using document root <none> and password file <none>
I20260809 07:51:12.188138 14010 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:12.188282 14010 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:12.188740 14010 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:12.192255 14010 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "a533c201e39c434899abc9487d1d5c8c"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:12.193364 14010 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "a533c201e39c434899abc9487d1d5c8c"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:12.200565 14010 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.002s sys 0.003s
I20260809 07:51:12.205499 14029 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:12.207285 14010 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:51:12.207468 14010 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "a533c201e39c434899abc9487d1d5c8c"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:12.207720 14010 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:12.228840 14010 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:12.232985 14010 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:12.233291 14010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:12.257907 14010 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:40109
I20260809 07:51:12.257917 14102 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:40109 every 8 connection(s)
I20260809 07:51:12.259902 14010 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:12.265125 14103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:12.268293 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14010
I20260809 07:51:12.268550 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:51:12.274415 14103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:12 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
I20260809 07:51:12.302624 14103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:12.304148 14036 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37063: connect: Connection refused (error 111)
W20260809 07:51:12.305933 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37063: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37063: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
I20260809 07:51:12.355418 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } attempt: 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
W20260809 07:51:12.357640 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37063: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37063: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:12.388477 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37063
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:12.425873 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } attempt: 2
W20260809 07:51:12.427979 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37063: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37063: connect: Connection refused (error 111)
I20260809 07:51:12.519243 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } attempt: 3
W20260809 07:51:12.521426 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37063: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37063: connect: Connection refused (error 111)
I20260809 07:51:12.664749 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } attempt: 4
W20260809 07:51:12.667007 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37063: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37063: connect: Connection refused (error 111)
W20260809 07:51:12.735248 14116 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:12.735556 14116 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:12.735646 14116 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:12.743556 14116 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:12.743652 14116 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:12.743706 14116 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:12.743755 14116 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:12.754189 14116 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:12.755736 14116 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:12.757067 14116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:12.765260 14121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:12.765304 14122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:12.765967 14124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:12.766870 14116 server_base.cc:1061] running on GCE node
I20260809 07:51:12.767540 14116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:12.767683 14116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:12.767746 14116 hybrid_clock.cc:648] HybridClock initialized: now 1786261872767729 us; error 0 us; skew 500 ppm
I20260809 07:51:12.768039 14116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:12 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:12.771416 14116 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:51:12.773279 14116 webserver.cc:533] Webserver started at http://127.26.37.61:34881/ using document root <none> and password file <none>
I20260809 07:51:12.774017 14116 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:12.774185 14116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:12.774564 14116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:12.777004 14116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "376a94640d2d4012bdcfadec09302ab3"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:12.777819 14116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "376a94640d2d4012bdcfadec09302ab3"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:12.783421 14116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:51:12.787076 14132 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:12.788554 14116 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:51:12.788729 14116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "376a94640d2d4012bdcfadec09302ab3"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:12.788928 14116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:12.803784 14116 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:12.807579 14116 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:12.807869 14116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:12.827791 14116 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:37063
I20260809 07:51:12.827814 14202 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:37063 every 8 connection(s)
I20260809 07:51:12.829315 14116 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:51:12.833691 14203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:12.834017 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14116
I20260809 07:51:12.834223 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:51:12.839249 14203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:12 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
I20260809 07:51:12.860744 14203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
Aug 09 07:51:12 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:12.876916 14203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:12.878257 14134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
W20260809 07:51:12.880026 14203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:51:12.929454 14203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 1
W20260809 07:51:12.931826 14203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:12.944403 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:43251
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:12.966295 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } attempt: 5
Aug 09 07:51:12 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:12.980439 14103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:12.982743 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
I20260809 07:51:13.000065 14203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 2
W20260809 07:51:13.002475 14203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
I20260809 07:51:13.033978 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 1
W20260809 07:51:13.036084 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
I20260809 07:51:13.093766 14203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 3
W20260809 07:51:13.095976 14203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
I20260809 07:51:13.104367 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 2
W20260809 07:51:13.106276 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
I20260809 07:51:13.212527 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 3
W20260809 07:51:13.214972 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
I20260809 07:51:13.239226 14203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 4
W20260809 07:51:13.241647 14203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
W20260809 07:51:13.306414 14219 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:13.306809 14219 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:13.306905 14219 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:13.315493 14219 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:13.315610 14219 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:13.315672 14219 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:13.315726 14219 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:13.326875 14219 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:43251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:13.328594 14219 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:13.330017 14219 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:13.338784 14225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:13.338897 14227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:13.339134 14230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:13.340157 14219 server_base.cc:1061] running on GCE node
I20260809 07:51:13.340947 14219 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:13.341141 14219 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:13.341255 14219 hybrid_clock.cc:648] HybridClock initialized: now 1786261873341216 us; error 0 us; skew 500 ppm
I20260809 07:51:13.341674 14219 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261873, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:13.345048 14219 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:51:13.346927 14219 webserver.cc:533] Webserver started at http://127.26.37.60:46455/ using document root <none> and password file <none>
I20260809 07:51:13.347651 14219 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:13.347787 14219 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:13.348156 14219 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:13.350725 14219 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "6ba678862f634122ab88fea23f1e7932"
format_stamp: "Formatted at 2026-08-09 07:51:13 on dist-test-slave-bw88"
I20260809 07:51:13.351542 14219 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "6ba678862f634122ab88fea23f1e7932"
format_stamp: "Formatted at 2026-08-09 07:51:13 on dist-test-slave-bw88"
I20260809 07:51:13.356853 14219 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 07:51:13.360385 14238 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:13.361846 14219 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:13.362046 14219 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "6ba678862f634122ab88fea23f1e7932"
format_stamp: "Formatted at 2026-08-09 07:51:13 on dist-test-slave-bw88"
I20260809 07:51:13.362270 14219 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:13.392228 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 4
W20260809 07:51:13.394550 14103 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111)
I20260809 07:51:13.397883 14219 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:13.405715 14219 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:13.406013 14219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:13.427666 14219 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:43251
I20260809 07:51:13.427690 14313 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:43251 every 8 connection(s)
I20260809 07:51:13.429323 14219 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:51:13.430452 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14219
I20260809 07:51:13.430680 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:51:13.435011 14314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:13.441517 14314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261873, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:13.465328 14314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261873, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:13.480393 14314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261873, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:13.500159 14314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Bootstrap starting.
I20260809 07:51:13.503433 14314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:13.504484 14314 log.cc:826] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:13.507375 14314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: No bootstrap required, opened a new log
I20260809 07:51:13.512440 14314 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.512877 14314 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:13.513176 14314 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba678862f634122ab88fea23f1e7932, State: Initialized, Role: FOLLOWER
I20260809 07:51:13.513938 14314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.514882 14333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:13.515245 14333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:13.515949 14314 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:51:13.526302 14348 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:13.526453 14348 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:13.523914 14219 master_runner.cc:187] Error getting master registration for 127.26.37.62:40109: OK, instance_id { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" instance_seqno: 1786261872250952 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:51:13.531572 14219 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:13.540933 14203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 5
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:13.561894 14203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: Bootstrap starting.
I20260809 07:51:13.566217 14203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:13.567870 14203 log.cc:826] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:13.571121 14203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: No bootstrap required, opened a new log
I20260809 07:51:13.578872 14203 raft_consensus.cc:359] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.579259 14203 raft_consensus.cc:385] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:13.579388 14203 raft_consensus.cc:740] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376a94640d2d4012bdcfadec09302ab3, State: Initialized, Role: FOLLOWER
I20260809 07:51:13.580206 14203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.581511 14351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:13.581926 14351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:13.582854 14203 sys_catalog.cc:565] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:51:13.593092 14366 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:13.593281 14366 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:13.590359 14116 master_runner.cc:187] Error getting master registration for 127.26.37.62:40109: OK, instance_id { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" instance_seqno: 1786261872250952 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:51:13.597724 14116 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:51:13.658356 14247 tablet.cc:2406] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:13.671794 14103 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } attempt: 5
Aug 09 07:51:13 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261872, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:13.695047 14103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap starting.
I20260809 07:51:13.699429 14103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:13.700960 14103 log.cc:826] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:13.704471 14103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: No bootstrap required, opened a new log
I20260809 07:51:13.714308 14103 raft_consensus.cc:359] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.714685 14103 raft_consensus.cc:385] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:13.714854 14103 raft_consensus.cc:740] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a533c201e39c434899abc9487d1d5c8c, State: Initialized, Role: FOLLOWER
I20260809 07:51:13.715626 14103 consensus_queue.cc:260] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.716703 14370 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:13.717129 14370 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:13.718175 14103 sys_catalog.cc:565] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:51:13.737110 14038 tablet.cc:2406] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:13.739553 14010 master_runner.cc:264] No leader master found from master a533c201e39c434899abc9487d1d5c8c
I20260809 07:51:13.739692 14010 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:51:13.739936 14382 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:13.740146 14382 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 07:51:13.810616 14140 tablet.cc:2406] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:13.850636 14333 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:13.850970 14333 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.852934 14333 leader_election.cc:290] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a533c201e39c434899abc9487d1d5c8c (127.26.37.62:40109), 376a94640d2d4012bdcfadec09302ab3 (127.26.37.61:37063)
I20260809 07:51:13.854624 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ba678862f634122ab88fea23f1e7932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a533c201e39c434899abc9487d1d5c8c" is_pre_election: true
I20260809 07:51:13.855247 14073 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba678862f634122ab88fea23f1e7932 in term 0.
I20260809 07:51:13.855916 14169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ba678862f634122ab88fea23f1e7932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376a94640d2d4012bdcfadec09302ab3" is_pre_election: true
I20260809 07:51:13.856395 14169 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba678862f634122ab88fea23f1e7932 in term 0.
I20260809 07:51:13.857296 14245 leader_election.cc:304] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 376a94640d2d4012bdcfadec09302ab3, 6ba678862f634122ab88fea23f1e7932; no voters:
I20260809 07:51:13.858379 14333 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:51:13.858534 14333 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:13.858647 14333 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:13.862138 14333 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.863360 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ba678862f634122ab88fea23f1e7932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a533c201e39c434899abc9487d1d5c8c"
I20260809 07:51:13.863595 14073 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:13.867434 14073 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba678862f634122ab88fea23f1e7932 in term 1.
I20260809 07:51:13.868009 14239 leader_election.cc:304] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ba678862f634122ab88fea23f1e7932, a533c201e39c434899abc9487d1d5c8c; no voters:
I20260809 07:51:13.872072 14169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ba678862f634122ab88fea23f1e7932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376a94640d2d4012bdcfadec09302ab3"
I20260809 07:51:13.872314 14169 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:13.875762 14169 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba678862f634122ab88fea23f1e7932 in term 1.
I20260809 07:51:13.878541 14333 leader_election.cc:290] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [CANDIDATE]: Term 1 election: Requested vote from peers a533c201e39c434899abc9487d1d5c8c (127.26.37.62:40109), 376a94640d2d4012bdcfadec09302ab3 (127.26.37.61:37063)
I20260809 07:51:13.879050 14333 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:13.879514 14333 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 1 LEADER]: Becoming Leader. State: Replica: 6ba678862f634122ab88fea23f1e7932, State: Running, Role: LEADER
I20260809 07:51:13.880301 14333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:13.895046 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ba678862f634122ab88fea23f1e7932. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:13.895486 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:13.914115 14388 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:51:13.916905 14388 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:51:13.938624 14169 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 1 FOLLOWER]: Refusing update from remote peer 6ba678862f634122ab88fea23f1e7932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:13.939857 14386 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:13.947319 14073 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Refusing update from remote peer 6ba678862f634122ab88fea23f1e7932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:13.948374 14386 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:13.984674 14370 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ba678862f634122ab88fea23f1e7932. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:13.984948 14370 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:13.988381 14389 mvcc.cc:204] Tried to move back new op lower bound from 7316528635641217024 to 7316528635427872768. Current Snapshot: MvccSnapshot[applied={T|T < 7316528635641217024}]
I20260809 07:51:13.992851 14391 mvcc.cc:204] Tried to move back new op lower bound from 7316528635641217024 to 7316528635427872768. Current Snapshot: MvccSnapshot[applied={T|T < 7316528635641217024}]
I20260809 07:51:14.001986 14370 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:14.002233 14370 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:14.006891 14351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ba678862f634122ab88fea23f1e7932. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:14.007174 14351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:14.009449 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:14.009722 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:14.010085 14388 catalog_manager.cc:1383] Generated new cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:14.010197 14388 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:51:14.013659 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:14.013886 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:14.016089 14390 mvcc.cc:204] Tried to move back new op lower bound from 7316528635641217024 to 7316528635427872768. Current Snapshot: MvccSnapshot[applied={T|T < 7316528635641217024}]
I20260809 07:51:14.035321 14351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:14.035580 14351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:14.061882 14388 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:51:14.063473 14388 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:51:14.078579 14388 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Generated new TSK 0
I20260809 07:51:14.079775 14388 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:14.236641 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:14.595237 14366 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:14.595376 14366 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: loading cluster ID for follower catalog manager: success
I20260809 07:51:14.598273 14366 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: acquiring CA information for follower catalog manager: success
I20260809 07:51:14.600270 14366 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:51:14.607411 14409 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:14.607756 14409 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:14.607904 14409 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:14.616693 14409 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:14.616978 14409 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:14.628000 14409 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:14.629715 14409 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:14.631100 14409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:14.641309 14417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:14.641412 14418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:14.642056 14421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:14.642357 14409 server_base.cc:1061] running on GCE node
I20260809 07:51:14.642978 14409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:14.643128 14409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:14.643194 14409 hybrid_clock.cc:648] HybridClock initialized: now 1786261874643177 us; error 0 us; skew 500 ppm
I20260809 07:51:14.643452 14409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:14 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261874, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:14.646893 14409 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:14.648404 14409 webserver.cc:533] Webserver started at http://127.26.37.1:37265/ using document root <none> and password file <none>
I20260809 07:51:14.649098 14409 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:14.649191 14409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:14.649556 14409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:14.651985 14409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:14.652714 14409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:14.666530 14409 fs_manager.cc:696] Time spent creating directory manager: real 0.013s user 0.014s sys 0.001s
I20260809 07:51:14.670199 14428 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:14.671684 14409 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:14.671846 14409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:14.672039 14409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:14.693198 14409 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:14.697196 14409 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:14.697533 14409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:14.698451 14409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:14.700083 14409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:14.700184 14409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:14.700326 14409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:14.700393 14409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:14.732051 14409 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35721
I20260809 07:51:14.732131 14583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35721 every 8 connection(s)
I20260809 07:51:14.733544 14409 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:14.736198 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14409
I20260809 07:51:14.736452 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:51:14.742892 14382 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:14.743017 14382 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: loading cluster ID for follower catalog manager: success
I20260809 07:51:14.747727 14382 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: acquiring CA information for follower catalog manager: success
I20260809 07:51:14.753511 14382 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:51:14 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261874, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:14 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261874, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:14 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261874, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:14.775504 14587 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37063
I20260809 07:51:14.775928 14587 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:14.775935 14584 heartbeater.cc:344] Connected to a master server at 127.26.37.60:43251
I20260809 07:51:14.776093 14584 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:14.776511 14585 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:14.776655 14585 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:14.777200 14584 heartbeater.cc:507] Master 127.26.37.60:43251 requested a full tablet report, sending...
I20260809 07:51:14.777240 14585 heartbeater.cc:507] Master 127.26.37.62:40109 requested a full tablet report, sending...
I20260809 07:51:14.777662 14587 heartbeater.cc:507] Master 127.26.37.61:37063 requested a full tablet report, sending...
I20260809 07:51:14.780051 14056 ts_manager.cc:194] Registered new tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:35721)
I20260809 07:51:14.780179 14153 ts_manager.cc:194] Registered new tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:35721)
I20260809 07:51:14.780524 14261 ts_manager.cc:194] Registered new tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:35721)
I20260809 07:51:14.782836 14261 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:52093
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:14.867074 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:15.214677 14605 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:15.214977 14605 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:15.215129 14605 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:15.223774 14605 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:15.223935 14605 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:51:15.233745 14605 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:15.235189 14605 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:15.236445 14605 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:15.245891 14611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:15.245826 14613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:15.246523 14616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:15.247376 14605 server_base.cc:1061] running on GCE node
I20260809 07:51:15.247843 14605 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:15.247991 14605 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:15.248054 14605 hybrid_clock.cc:648] HybridClock initialized: now 1786261875248037 us; error 0 us; skew 500 ppm
I20260809 07:51:15.248284 14605 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:15 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:15.251458 14605 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:51:15.252923 14605 webserver.cc:533] Webserver started at http://127.26.37.2:45749/ using document root <none> and password file <none>
I20260809 07:51:15.253695 14605 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:15.253830 14605 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:15.254169 14605 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:15.256434 14605 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "b44794667c9d4a8f9cab88bf9746a495"
format_stamp: "Formatted at 2026-08-09 07:51:15 on dist-test-slave-bw88"
I20260809 07:51:15.257539 14605 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "b44794667c9d4a8f9cab88bf9746a495"
format_stamp: "Formatted at 2026-08-09 07:51:15 on dist-test-slave-bw88"
I20260809 07:51:15.262706 14605 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 07:51:15.265997 14625 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:15.267297 14605 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:15.267480 14605 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "b44794667c9d4a8f9cab88bf9746a495"
format_stamp: "Formatted at 2026-08-09 07:51:15 on dist-test-slave-bw88"
I20260809 07:51:15.267673 14605 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:15.282528 14605 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:15.286193 14605 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:15.286495 14605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:15.287441 14605 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:15.289059 14605 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:15.289131 14605 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:15.289218 14605 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:15.289261 14605 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:15.324319 14605 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:44215
I20260809 07:51:15.324410 14780 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:44215 every 8 connection(s)
I20260809 07:51:15.325729 14605 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:15.334228 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14605
I20260809 07:51:15.334471 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:51:15 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:15 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:15 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:15.362236 14781 heartbeater.cc:344] Connected to a master server at 127.26.37.60:43251
I20260809 07:51:15.362262 14784 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37063
I20260809 07:51:15.362471 14783 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:15.362689 14784 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:15.362689 14783 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:15.362754 14781 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:15.363977 14781 heartbeater.cc:507] Master 127.26.37.60:43251 requested a full tablet report, sending...
I20260809 07:51:15.364046 14783 heartbeater.cc:507] Master 127.26.37.62:40109 requested a full tablet report, sending...
I20260809 07:51:15.364073 14784 heartbeater.cc:507] Master 127.26.37.61:37063 requested a full tablet report, sending...
I20260809 07:51:15.365902 14261 ts_manager.cc:194] Registered new tserver with Master: b44794667c9d4a8f9cab88bf9746a495 (127.26.37.2:44215)
I20260809 07:51:15.366133 14056 ts_manager.cc:194] Registered new tserver with Master: b44794667c9d4a8f9cab88bf9746a495 (127.26.37.2:44215)
I20260809 07:51:15.366137 14153 ts_manager.cc:194] Registered new tserver with Master: b44794667c9d4a8f9cab88bf9746a495 (127.26.37.2:44215)
I20260809 07:51:15.367038 14261 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:37467
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:15.455899 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:15.785480 14584 heartbeater.cc:499] Master 127.26.37.60:43251 was elected leader, sending a full tablet report...
W20260809 07:51:15.805081 14801 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:15.805418 14801 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:15.805536 14801 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:15.813849 14801 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:15.814033 14801 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:51:15.824607 14801 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:15.826114 14801 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:15.827445 14801 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:15.836956 14807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:15.836921 14806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:15.838084 14809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:15.839105 14801 server_base.cc:1061] running on GCE node
I20260809 07:51:15.839605 14801 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:15.839737 14801 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:15.839818 14801 hybrid_clock.cc:648] HybridClock initialized: now 1786261875839782 us; error 0 us; skew 500 ppm
I20260809 07:51:15.840104 14801 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:15 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:15.843520 14801 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:51:15.845167 14801 webserver.cc:533] Webserver started at http://127.26.37.3:33863/ using document root <none> and password file <none>
I20260809 07:51:15.845932 14801 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:15.846067 14801 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:15.846395 14801 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:15.848776 14801 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "8fee1a9e44fe49769b45ebf615d0c8b5"
format_stamp: "Formatted at 2026-08-09 07:51:15 on dist-test-slave-bw88"
I20260809 07:51:15.849633 14801 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "8fee1a9e44fe49769b45ebf615d0c8b5"
format_stamp: "Formatted at 2026-08-09 07:51:15 on dist-test-slave-bw88"
I20260809 07:51:15.854880 14801 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:51:15.858177 14817 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:15.859514 14801 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:15.859688 14801 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "8fee1a9e44fe49769b45ebf615d0c8b5"
format_stamp: "Formatted at 2026-08-09 07:51:15 on dist-test-slave-bw88"
I20260809 07:51:15.859879 14801 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:15.884090 14801 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:15.887862 14801 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:15.888152 14801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:15.889168 14801 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:15.890789 14801 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:15.890865 14801 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:15.890944 14801 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:15.890986 14801 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:15.927042 14801 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:35173
I20260809 07:51:15.927139 14972 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:35173 every 8 connection(s)
I20260809 07:51:15.928440 14801 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:15.931608 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14801
I20260809 07:51:15.931862 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:51:15 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:15 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:15 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261875, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:15.962189 14974 heartbeater.cc:344] Connected to a master server at 127.26.37.60:43251
I20260809 07:51:15.962648 14974 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:15.963232 14975 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:15.963425 14975 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:15.964154 14974 heartbeater.cc:507] Master 127.26.37.60:43251 requested a full tablet report, sending...
I20260809 07:51:15.964180 14976 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37063
I20260809 07:51:15.964360 14976 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:15.964869 14976 heartbeater.cc:507] Master 127.26.37.61:37063 requested a full tablet report, sending...
I20260809 07:51:15.965431 14975 heartbeater.cc:507] Master 127.26.37.62:40109 requested a full tablet report, sending...
I20260809 07:51:15.966120 14261 ts_manager.cc:194] Registered new tserver with Master: 8fee1a9e44fe49769b45ebf615d0c8b5 (127.26.37.3:35173)
I20260809 07:51:15.966358 14153 ts_manager.cc:194] Registered new tserver with Master: 8fee1a9e44fe49769b45ebf615d0c8b5 (127.26.37.3:35173)
I20260809 07:51:15.966650 14056 ts_manager.cc:194] Registered new tserver with Master: 8fee1a9e44fe49769b45ebf615d0c8b5 (127.26.37.3:35173)
I20260809 07:51:15.967061 14261 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:59987
I20260809 07:51:15.971761 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:15.978646 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:15.978801 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14010
I20260809 07:51:15.979691 14097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:16.088477 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14010
I20260809 07:51:16.117187 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40109
--webserver_interface=127.26.37.62
--webserver_port=33779
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:16.369280 14783 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:40109 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:40109: connect: Connection refused (error 111)
I20260809 07:51:16.369727 14781 heartbeater.cc:499] Master 127.26.37.60:43251 was elected leader, sending a full tablet report...
W20260809 07:51:16.470295 14988 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:16.470703 14988 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:16.470809 14988 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:16.480059 14988 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:16.480199 14988 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:16.480311 14988 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:16.480391 14988 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:16.491142 14988 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=33779
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:16.492765 14988 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:16.494302 14988 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:16.502676 14995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:16.503232 14994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:16.503705 14997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:16.504149 14988 server_base.cc:1061] running on GCE node
I20260809 07:51:16.504824 14988 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:16.504974 14988 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:16.505040 14988 hybrid_clock.cc:648] HybridClock initialized: now 1786261876505023 us; error 0 us; skew 500 ppm
I20260809 07:51:16.505326 14988 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:16 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261876, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:16.509050 14988 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:16.511183 14988 webserver.cc:533] Webserver started at http://127.26.37.62:33779/ using document root <none> and password file <none>
I20260809 07:51:16.511895 14988 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:16.512020 14988 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:16.517593 14988 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:51:16.520699 15005 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:16.522176 14988 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:16.522315 14988 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "a533c201e39c434899abc9487d1d5c8c"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:16.523034 14988 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:16.549767 14988 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:16.553942 14988 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:16.554173 14988 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260809 07:51:16.565402 14239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:40109: connect: Connection refused (error 111)
W20260809 07:51:16.567696 14239 consensus_peers.cc:597] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 -> Peer a533c201e39c434899abc9487d1d5c8c (127.26.37.62:40109): Couldn't send request to peer a533c201e39c434899abc9487d1d5c8c. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:40109: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:51:16.578130 14988 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:40109
I20260809 07:51:16.578137 15081 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:40109 every 8 connection(s)
I20260809 07:51:16.579813 14988 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:16.582195 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14988
I20260809 07:51:16.582594 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14409
I20260809 07:51:16.588841 15082 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:16.592846 15082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap starting.
Aug 09 07:51:16 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:51:16.601238 14576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:16.613940 15082 log.cc:826] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:16.623713 15082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:16.624310 15082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap complete.
I20260809 07:51:16.632035 15082 raft_consensus.cc:359] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:16.632809 15082 raft_consensus.cc:740] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a533c201e39c434899abc9487d1d5c8c, State: Initialized, Role: FOLLOWER
I20260809 07:51:16.633664 15082 consensus_queue.cc:260] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:16.634716 15091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:16.635049 15091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:16.635926 15082 sys_catalog.cc:565] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:16.654150 15105 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:16.654374 15105 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: loading cluster ID for follower catalog manager: success
I20260809 07:51:16.658596 15105 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: acquiring CA information for follower catalog manager: success
Aug 09 07:51:16 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261876, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:16.661484 15105 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:16.706632 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14409
I20260809 07:51:16.732054 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35721
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--webserver_port=37265
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:16.969739 14974 heartbeater.cc:499] Master 127.26.37.60:43251 was elected leader, sending a full tablet report...
I20260809 07:51:16.981251 15029 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8fee1a9e44fe49769b45ebf615d0c8b5" instance_seqno: 1786261875918604) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:50301; Asking this server to re-register.
I20260809 07:51:16.982203 14975 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:16.984229 15029 ts_manager.cc:194] Registered new tserver with Master: 8fee1a9e44fe49769b45ebf615d0c8b5 (127.26.37.3:35173)
I20260809 07:51:17.023515 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:17.024041 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:17.024794 15091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ba678862f634122ab88fea23f1e7932. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:17.025091 15091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:51:17.100989 15114 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:17.101357 15114 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:17.101531 15114 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:17.110316 15114 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:17.110555 15114 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:17.121593 15114 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=37265
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:17.123242 15114 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:17.124624 15114 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:17.134388 15127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:17.134387 15126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:17.135465 15129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:17.136085 15114 server_base.cc:1061] running on GCE node
I20260809 07:51:17.136682 15114 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:17.136826 15114 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:17.136893 15114 hybrid_clock.cc:648] HybridClock initialized: now 1786261877136872 us; error 0 us; skew 500 ppm
I20260809 07:51:17.137185 15114 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:17 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261877, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:17.140736 15114 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:17.142513 15114 webserver.cc:533] Webserver started at http://127.26.37.1:37265/ using document root <none> and password file <none>
I20260809 07:51:17.143258 15114 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:17.143394 15114 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:17.149219 15114 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:51:17.152069 15138 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:17.153455 15114 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:17.153630 15114 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:17.154367 15114 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:17.180224 15114 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:17.184229 15114 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:17.184522 15114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:17.185551 15114 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:17.187309 15114 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:17.187413 15114 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:17.187536 15114 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:17.187633 15114 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:17.223522 15114 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35721
I20260809 07:51:17.223628 15299 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35721 every 8 connection(s)
I20260809 07:51:17.225037 15114 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:17.231518 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15114
Aug 09 07:51:17 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261877, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:17 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261877, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:17 dist-test-slave-bw88 krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261877, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:17.256135 15303 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37063
I20260809 07:51:17.256151 15302 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:17.256507 15303 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:17.256518 15302 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:17.257531 15303 heartbeater.cc:507] Master 127.26.37.61:37063 requested a full tablet report, sending...
I20260809 07:51:17.257616 15302 heartbeater.cc:507] Master 127.26.37.62:40109 requested a full tablet report, sending...
I20260809 07:51:17.259591 14153 ts_manager.cc:194] Re-registered known tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:35721)
I20260809 07:51:17.259707 15029 ts_manager.cc:194] Registered new tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:35721)
I20260809 07:51:17.261372 15300 heartbeater.cc:344] Connected to a master server at 127.26.37.60:43251
I20260809 07:51:17.261535 15300 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:17.262064 15300 heartbeater.cc:507] Master 127.26.37.60:43251 requested a full tablet report, sending...
I20260809 07:51:17.263355 14261 ts_manager.cc:194] Re-registered known tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:35721)
I20260809 07:51:17.264575 14261 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:60853
I20260809 07:51:17.285506 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15114
W20260809 07:51:17.285933 15314 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:17.282275 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.26.37.1:35721)
0809 07:51:17.283016 (+ 741us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:17.283024 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:17.283059 (+ 35us) server_negotiation.cc:408] Connection header received
0809 07:51:17.283159 (+ 100us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:17.283166 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:17.283381 (+ 215us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:17.283554 (+ 173us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:17.284810 (+ 1256us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34876: BlockingRecv error: recv got EOF from 127.0.0.1:34876 (error 108)
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":382,"threads_started":1}
W20260809 07:51:17.287746 15314 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:17.286287 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:34880 (local address 127.26.37.1:35721)
0809 07:51:17.286680 (+ 393us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:17.286684 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:17.286700 (+ 16us) server_negotiation.cc:408] Connection header received
0809 07:51:17.286776 (+ 76us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:17.286781 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:17.286856 (+ 75us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:17.286965 (+ 109us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:17.287580 (+ 615us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34880: BlockingRecv error: recv got EOF from 127.0.0.1:34880 (error 108)
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":481,"threads_started":1}
W20260809 07:51:17.287717 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15114: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:35721: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:17.287863 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15114
I20260809 07:51:17.312517 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35721
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--webserver_port=37265
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:17.377527 14783 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:17.378578 15029 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b44794667c9d4a8f9cab88bf9746a495" instance_seqno: 1786261875315801) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:46535; Asking this server to re-register.
I20260809 07:51:17.379192 14783 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:17.380239 15029 ts_manager.cc:194] Registered new tserver with Master: b44794667c9d4a8f9cab88bf9746a495 (127.26.37.2:44215)
I20260809 07:51:17.608775 15117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:17.609066 15117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: LEADER
W20260809 07:51:17.667863 15317 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:17.668252 15317 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:17.668411 15317 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:51:17.668509 15317 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:17.676707 15317 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:17.676977 15317 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:17.687529 15317 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=37265
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:17.689142 15317 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:17.690579 15317 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:17.700603 15333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:17.700598 15334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:17.700855 15336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:17.701637 15317 server_base.cc:1061] running on GCE node
I20260809 07:51:17.702189 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:17.702350 15317 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:17.702473 15317 hybrid_clock.cc:648] HybridClock initialized: now 1786261877702412 us; error 0 us; skew 500 ppm
I20260809 07:51:17.702778 15317 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:17 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261877, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:17.706126 15317 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:17.707659 15317 webserver.cc:533] Webserver started at http://127.26.37.1:37265/ using document root <none> and password file <none>
I20260809 07:51:17.708366 15317 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:17.708523 15317 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:17.713967 15317 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:51:17.716519 15342 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:17.717892 15317 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:17.718071 15317 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:17.718750 15317 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:17.734894 15317 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:17.739040 15317 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:17.739352 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:17.740309 15317 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:17.741914 15317 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:17.742012 15317 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:17.742143 15317 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:17.742215 15317 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:51:17.742595 15317 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:51:17.742681 15317 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:51:17.742756 15317 fault_injection.cc:59] injecting fault for kudu (pid 15317): FLAGS_fault_before_start (process will exit)
I20260809 07:51:17.769176 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35721
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--webserver_port=37265
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:18.115429 15353 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:18.115751 15353 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:18.115844 15353 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:18.124336 15353 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:18.124509 15353 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:18.134742 15353 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=37265
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:18.136241 15353 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:18.137666 15353 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:18.147470 15358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:18.147919 15359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:18.148237 15363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:18.148561 15353 server_base.cc:1061] running on GCE node
I20260809 07:51:18.149169 15353 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:18.149364 15353 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:18.149469 15353 hybrid_clock.cc:648] HybridClock initialized: now 1786261878149434 us; error 0 us; skew 500 ppm
I20260809 07:51:18.149765 15353 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:18 dist-test-slave-bw88 krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261878, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:18.153010 15353 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:18.154595 15353 webserver.cc:533] Webserver started at http://127.26.37.1:37265/ using document root <none> and password file <none>
I20260809 07:51:18.155349 15353 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:18.155500 15353 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:18.160591 15353 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:51:18.163378 15370 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:18.164645 15353 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:18.164819 15353 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:18.165675 15353 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:18.183781 15353 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:18.187422 15353 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:18.187674 15353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:18.188659 15353 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:18.190284 15353 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:18.190414 15353 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:18.190552 15353 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:18.190636 15353 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:18.225174 15353 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35721
I20260809 07:51:18.225255 15521 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35721 every 8 connection(s)
I20260809 07:51:18.226742 15353 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:18.235219 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15353
I20260809 07:51:18.235643 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15353
W20260809 07:51:18.259285 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15353: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:35721: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:18.259528 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15353
I20260809 07:51:18.285597 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14605
W20260809 07:51:18.286190 15528 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:18.230506 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:59255 (local address 127.26.37.60:43251)
0809 07:51:18.254124 (+ 23618us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:18.254133 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:18.258397 (+ 4264us) server_negotiation.cc:408] Connection header received
0809 07:51:18.259334 (+ 937us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:18.259341 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:18.259420 (+ 79us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:18.259544 (+ 124us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:18.285208 (+ 25664us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:59255: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":23469,"thread_start_us":23355,"threads_started":1}
W20260809 07:51:18.286211 15526 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:18.230174 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:51761 (local address 127.26.37.62:40109)
0809 07:51:18.257109 (+ 26935us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:18.257118 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:18.257143 (+ 25us) server_negotiation.cc:408] Connection header received
0809 07:51:18.257226 (+ 83us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:18.257234 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:18.257360 (+ 126us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:18.258267 (+ 907us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:18.285222 (+ 26955us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:51761: BlockingRecv error: recv got EOF from 127.26.37.1:51761 (error 108)
Metrics: {"server-negotiator.queue_time_us":26783,"thread_start_us":26642,"threads_started":1}
W20260809 07:51:18.287002 15529 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:18.230845 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:48589 (local address 127.26.37.61:37063)
0809 07:51:18.253749 (+ 22904us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:18.253758 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:18.256534 (+ 2776us) server_negotiation.cc:408] Connection header received
0809 07:51:18.257167 (+ 633us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:18.257177 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:18.257291 (+ 114us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:18.257445 (+ 154us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:18.285218 (+ 27773us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:48589: BlockingRecv error: recv got EOF from 127.26.37.1:48589 (error 108)
Metrics: {"server-negotiator.queue_time_us":22762,"thread_start_us":22647,"threads_started":1}
W20260809 07:51:18.288735 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14605: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:44215: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:18.288906 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14605
I20260809 07:51:18.311334 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14801
W20260809 07:51:18.314280 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14801: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:35173: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:18.314422 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14801
I20260809 07:51:18.337668 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14988
W20260809 07:51:18.340097 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14988: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:40109: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:51:18.340193 15526 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:18.338486 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.26.37.62:40109)
0809 07:51:18.338718 (+ 232us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:18.338724 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:18.338838 (+ 114us) server_negotiation.cc:408] Connection header received
0809 07:51:18.339071 (+ 233us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:18.339081 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:18.339162 (+ 81us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:18.339292 (+ 130us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:18.340000 (+ 708us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51344: BlockingRecv error: recv got EOF from 127.0.0.1:51344 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 07:51:18.340340 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14988
I20260809 07:51:18.365135 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14116
I20260809 07:51:18.366294 14197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:18.475870 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14116
I20260809 07:51:18.505539 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14219
I20260809 07:51:18.506668 14307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:18.618657 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14219
I20260809 07:51:18.618618 14219 master_runner.cc:187] Error getting master registration for 127.26.37.62:40109: Network error: Client connection negotiation failed: client connection to 127.26.37.62:40109: connect: Connection refused (error 111),
Aug 09 07:51:18 dist-test-slave-bw88 krb5kdc[15546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:18 dist-test-slave-bw88 krb5kdc[15546](info): set up 2 sockets
Aug 09 07:51:18 dist-test-slave-bw88 krb5kdc[15546](info): commencing operation
krb5kdc: starting...
W20260809 07:51:20.778039 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.125s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:51:20 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:21.012699 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40109
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:21.364411 15565 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:21.364820 15565 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:21.364981 15565 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:21.372954 15565 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:21.373116 15565 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:21.373212 15565 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:21.373301 15565 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:21.383817 15565 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:21.385561 15565 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:21.386999 15565 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:21.395386 15572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:21.395428 15574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:21.396013 15576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:21.397032 15565 server_base.cc:1061] running on GCE node
I20260809 07:51:21.397809 15565 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:21.397955 15565 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:21.398022 15565 hybrid_clock.cc:648] HybridClock initialized: now 1786261881398002 us; error 0 us; skew 500 ppm
I20260809 07:51:21.398350 15565 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:21 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261881, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:21.401994 15565 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:21.403923 15565 webserver.cc:533] Webserver started at http://127.26.37.62:37349/ using document root <none> and password file <none>
I20260809 07:51:21.404676 15565 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:21.404817 15565 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:21.410346 15565 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:51:21.413249 15588 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:21.414655 15565 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:21.414841 15565 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "a533c201e39c434899abc9487d1d5c8c"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:21.415619 15565 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:21.437623 15565 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:21.445333 15565 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:21.445686 15565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:21.466305 15565 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:40109
I20260809 07:51:21.466322 15657 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:40109 every 8 connection(s)
I20260809 07:51:21.467919 15565 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:21.474893 15658 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:21.477285 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15565
I20260809 07:51:21.477550 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:51:21.478935 15658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap starting.
I20260809 07:51:21.500372 15658 log.cc:826] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:21.512140 15658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:21.512831 15658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap complete.
I20260809 07:51:21.520649 15658 raft_consensus.cc:359] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:21.521510 15658 raft_consensus.cc:740] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a533c201e39c434899abc9487d1d5c8c, State: Initialized, Role: FOLLOWER
I20260809 07:51:21.522264 15658 consensus_queue.cc:260] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:21.523190 15666 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:21.523509 15666 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:21.524326 15658 sys_catalog.cc:565] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:21.535317 15687 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:21.535429 15687 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: loading cluster ID for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:21.543076 15687 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: acquiring CA information for follower catalog manager: success
Aug 09 07:51:21 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261881, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:21.545735 15687 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:21.561331 15565 master_runner.cc:187] Error getting master registration for 127.26.37.61:37063: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37063: connect: Connection refused (error 111),
I20260809 07:51:21.567430 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:21.597723 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37063
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:21.945878 15694 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:21.946257 15694 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:21.946403 15694 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:21.954677 15694 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:21.954830 15694 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:21.954945 15694 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:21.955196 15694 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:21.965943 15694 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:21.967648 15694 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:21.969055 15694 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:21.977494 15702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:21.977468 15701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:21.978205 15708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:21.978703 15694 server_base.cc:1061] running on GCE node
I20260809 07:51:21.979401 15694 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:21.979592 15694 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:21.979691 15694 hybrid_clock.cc:648] HybridClock initialized: now 1786261881979664 us; error 0 us; skew 500 ppm
I20260809 07:51:21.980026 15694 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:21 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261881, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:21.988303 15694 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:51:21.990306 15694 webserver.cc:533] Webserver started at http://127.26.37.61:43583/ using document root <none> and password file <none>
I20260809 07:51:21.991045 15694 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:21.991189 15694 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:21.996516 15694 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 07:51:21.999620 15718 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:22.001029 15694 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:22.001211 15694 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "376a94640d2d4012bdcfadec09302ab3"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:22.002087 15694 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:22.018199 15694 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:22.022431 15694 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:22.022766 15694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:22.043644 15694 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:37063
I20260809 07:51:22.043665 15792 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:37063 every 8 connection(s)
I20260809 07:51:22.045307 15694 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:51:22.052340 15793 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:22.054530 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15694
I20260809 07:51:22.054716 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:51:22.055740 15793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: Bootstrap starting.
I20260809 07:51:22.074105 15793 log.cc:826] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:22.085645 15793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:22.086215 15793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: Bootstrap complete.
I20260809 07:51:22.093078 15793 raft_consensus.cc:359] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:22.093881 15793 raft_consensus.cc:740] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376a94640d2d4012bdcfadec09302ab3, State: Initialized, Role: FOLLOWER
I20260809 07:51:22.094770 15793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:22.095630 15799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:22.095897 15799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:22.097031 15793 sys_catalog.cc:565] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:22.108300 15816 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:22.108410 15816 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: loading cluster ID for follower catalog manager: success
I20260809 07:51:22.112962 15816 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: acquiring CA information for follower catalog manager: success
I20260809 07:51:22.115805 15816 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:22.143484 15694 master_runner.cc:187] Error getting master registration for 127.26.37.60:43251: Network error: Client connection negotiation failed: client connection to 127.26.37.60:43251: connect: Connection refused (error 111),
I20260809 07:51:22.149129 15694 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:22.178185 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:43251
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:22.565025 15826 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:22.565395 15826 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:22.565572 15826 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:22.573745 15826 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:22.573848 15826 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:22.573979 15826 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:22.574048 15826 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:22.585132 15826 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:43251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:22.586874 15826 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:22.588280 15826 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:22.596633 15832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:22.596983 15839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:22.597271 15835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:22.598289 15826 server_base.cc:1061] running on GCE node
I20260809 07:51:22.599036 15826 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:22.599220 15826 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:22.599311 15826 hybrid_clock.cc:648] HybridClock initialized: now 1786261882599289 us; error 0 us; skew 500 ppm
I20260809 07:51:22.599644 15826 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:22 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261882, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:22.603404 15826 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:51:22.605319 15826 webserver.cc:533] Webserver started at http://127.26.37.60:40073/ using document root <none> and password file <none>
I20260809 07:51:22.606066 15826 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:22.606206 15826 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:22.611673 15826 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 07:51:22.614620 15847 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:22.616003 15826 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:22.616179 15826 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "6ba678862f634122ab88fea23f1e7932"
format_stamp: "Formatted at 2026-08-09 07:51:13 on dist-test-slave-bw88"
I20260809 07:51:22.616907 15826 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:22.637399 15826 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:22.640961 15826 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:22.641266 15826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:22.662087 15826 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:43251
I20260809 07:51:22.662247 15920 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:43251 every 8 connection(s)
I20260809 07:51:22.664017 15826 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:51:22.666096 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15826
I20260809 07:51:22.666316 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:51:22.673931 15923 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:22.679239 15923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Bootstrap starting.
Aug 09 07:51:22 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:22 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:22.701695 15923 log.cc:826] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:22.712471 15923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:22.713183 15923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Bootstrap complete.
Aug 09 07:51:22 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:22.722335 15923 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:22.723397 15923 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba678862f634122ab88fea23f1e7932, State: Initialized, Role: FOLLOWER
I20260809 07:51:22.724182 15923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:22.725116 15931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:22.725497 15931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:22.726380 15923 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:22.738266 15954 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:22.738359 15954 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: loading cluster ID for follower catalog manager: success
I20260809 07:51:22.743990 15954 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: acquiring CA information for follower catalog manager: success
Aug 09 07:51:22 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261882, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:22.746621 15954 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:22.772159 15826 master_runner.cc:264] No leader master found from master 6ba678862f634122ab88fea23f1e7932
I20260809 07:51:22.772300 15826 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:23.186065 15958 raft_consensus.cc:493] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:23.186375 15958 raft_consensus.cc:515] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:23.188398 15958 leader_election.cc:290] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 376a94640d2d4012bdcfadec09302ab3 (127.26.37.61:37063), 6ba678862f634122ab88fea23f1e7932 (127.26.37.60:43251)
I20260809 07:51:23.196812 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a533c201e39c434899abc9487d1d5c8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "376a94640d2d4012bdcfadec09302ab3" is_pre_election: true
I20260809 07:51:23.197023 15886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a533c201e39c434899abc9487d1d5c8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ba678862f634122ab88fea23f1e7932" is_pre_election: true
I20260809 07:51:23.197461 15758 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a533c201e39c434899abc9487d1d5c8c in term 1.
I20260809 07:51:23.197574 15886 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a533c201e39c434899abc9487d1d5c8c in term 1.
I20260809 07:51:23.198086 15592 leader_election.cc:304] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 376a94640d2d4012bdcfadec09302ab3, a533c201e39c434899abc9487d1d5c8c; no voters:
I20260809 07:51:23.198791 15958 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:51:23.198954 15958 raft_consensus.cc:493] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:23.199080 15958 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:51:23.203025 15958 raft_consensus.cc:515] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:23.203670 15958 leader_election.cc:290] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [CANDIDATE]: Term 2 election: Requested vote from peers 376a94640d2d4012bdcfadec09302ab3 (127.26.37.61:37063), 6ba678862f634122ab88fea23f1e7932 (127.26.37.60:43251)
I20260809 07:51:23.204036 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a533c201e39c434899abc9487d1d5c8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "376a94640d2d4012bdcfadec09302ab3"
I20260809 07:51:23.204150 15886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a533c201e39c434899abc9487d1d5c8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ba678862f634122ab88fea23f1e7932"
I20260809 07:51:23.204222 15758 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:51:23.204371 15886 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:51:23.208096 15758 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a533c201e39c434899abc9487d1d5c8c in term 2.
I20260809 07:51:23.208168 15886 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a533c201e39c434899abc9487d1d5c8c in term 2.
I20260809 07:51:23.208544 15592 leader_election.cc:304] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 376a94640d2d4012bdcfadec09302ab3, a533c201e39c434899abc9487d1d5c8c; no voters:
I20260809 07:51:23.208830 15958 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:51:23.209277 15958 raft_consensus.cc:697] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 2 LEADER]: Becoming Leader. State: Replica: a533c201e39c434899abc9487d1d5c8c, State: Running, Role: LEADER
I20260809 07:51:23.209725 15958 consensus_queue.cc:237] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:23.212540 15962 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a533c201e39c434899abc9487d1d5c8c. Latest consensus state: current_term: 2 leader_uuid: "a533c201e39c434899abc9487d1d5c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:23.212833 15962 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:23.213615 15964 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:51:23.215622 15964 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:51:23.216424 15964 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:23.216508 15964 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:51:23.217648 15964 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:51:23.218425 15964 catalog_manager.cc:6103] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Loaded TSK: 0
I20260809 07:51:23.219269 15964 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:23.333004 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:23.613903 15758 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 2 FOLLOWER]: Refusing update from remote peer a533c201e39c434899abc9487d1d5c8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:51:23.615037 15962 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:51:23.619386 15970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a533c201e39c434899abc9487d1d5c8c. Latest consensus state: current_term: 2 leader_uuid: "a533c201e39c434899abc9487d1d5c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:23.619717 15970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:23.620934 15962 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a533c201e39c434899abc9487d1d5c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:23.621192 15962 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:23.622244 15970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a533c201e39c434899abc9487d1d5c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:23.622445 15970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:23.622524 15886 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 2 FOLLOWER]: Refusing update from remote peer a533c201e39c434899abc9487d1d5c8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:51:23.623400 15962 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:51:23.627050 15981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a533c201e39c434899abc9487d1d5c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:23.627377 15981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:23.627844 15977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a533c201e39c434899abc9487d1d5c8c. Latest consensus state: current_term: 2 leader_uuid: "a533c201e39c434899abc9487d1d5c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:23.628046 15977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:51:23.693769 15969 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:23.694114 15969 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:23.694270 15969 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:23.702886 15969 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:23.703119 15969 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:23.714083 15969 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:23.715776 15969 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:23.717195 15969 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:23.727348 15987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:23.727380 15988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:23.728024 15991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:23.728785 15969 server_base.cc:1061] running on GCE node
I20260809 07:51:23.729403 15969 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:23.729571 15969 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:23.729638 15969 hybrid_clock.cc:648] HybridClock initialized: now 1786261883729620 us; error 0 us; skew 500 ppm
I20260809 07:51:23.729889 15969 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:23 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:23.733500 15969 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:23.735019 15969 webserver.cc:533] Webserver started at http://127.26.37.1:39163/ using document root <none> and password file <none>
I20260809 07:51:23.735766 15969 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:23.735916 15969 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:23.741410 15969 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:23.744010 15998 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:23.745301 15969 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:51:23.745532 15969 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:23.746227 15969 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:23.785920 15969 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:23.789867 15969 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:23.790176 15969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:23.791188 15969 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:23.792874 15969 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:23.793004 15969 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:23.793151 15969 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:23.793243 15969 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:23.829761 15969 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46563
I20260809 07:51:23.829846 16168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46563 every 8 connection(s)
I20260809 07:51:23.831396 15969 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:23.839506 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15969
I20260809 07:51:23.839737 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:51:23 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:23 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:23 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261883, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:23.880393 16172 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37063
I20260809 07:51:23.880848 16172 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:23.880898 16169 heartbeater.cc:344] Connected to a master server at 127.26.37.60:43251
I20260809 07:51:23.881011 16169 heartbeater.cc:461] Registering TS with master...
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.2@KRBTEST.COM".
I20260809 07:51:23.880848 16171 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:23.881511 16171 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:23.882071 16172 heartbeater.cc:507] Master 127.26.37.61:37063 requested a full tablet report, sending...
I20260809 07:51:23.883303 16171 heartbeater.cc:507] Master 127.26.37.62:40109 requested a full tablet report, sending...
I20260809 07:51:23.884276 16169 heartbeater.cc:507] Master 127.26.37.60:43251 requested a full tablet report, sending...
I20260809 07:51:23.884980 15745 ts_manager.cc:194] Registered new tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:46563)
I20260809 07:51:23.884992 15609 ts_manager.cc:194] Registered new tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:46563)
I20260809 07:51:23.886000 15870 ts_manager.cc:194] Registered new tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:46563)
I20260809 07:51:23.887315 15609 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:53197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:23.964411 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:24.104189 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a533c201e39c434899abc9487d1d5c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:24.104611 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: LEADER
W20260809 07:51:24.313822 16188 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:24.314200 16188 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:24.314353 16188 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:24.322892 16188 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:24.323113 16188 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:51:24.333756 16188 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:24.335410 16188 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:24.336794 16188 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:24.346688 16195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:24.346736 16193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:24.347563 16197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:24.347848 16188 server_base.cc:1061] running on GCE node
I20260809 07:51:24.348317 16188 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:24.348460 16188 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:24.348521 16188 hybrid_clock.cc:648] HybridClock initialized: now 1786261884348505 us; error 0 us; skew 500 ppm
I20260809 07:51:24.348753 16188 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:24 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:24.352324 16188 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:51:24.353996 16188 webserver.cc:533] Webserver started at http://127.26.37.2:36189/ using document root <none> and password file <none>
I20260809 07:51:24.354691 16188 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:24.354797 16188 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:24.359859 16188 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:51:24.362669 16205 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:24.364166 16188 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:24.364305 16188 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "b44794667c9d4a8f9cab88bf9746a495"
format_stamp: "Formatted at 2026-08-09 07:51:15 on dist-test-slave-bw88"
I20260809 07:51:24.364988 16188 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:24.383852 16188 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:24.387476 16188 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:24.387715 16188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:24.388583 16188 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:24.390157 16188 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:24.390247 16188 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:24.390331 16188 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:24.390384 16188 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:24.426671 16188 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:41717
I20260809 07:51:24.426766 16355 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:41717 every 8 connection(s)
I20260809 07:51:24.428375 16188 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:24.429977 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16188
I20260809 07:51:24.430287 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:51:24 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:24 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:24 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:24.476315 16360 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37063
I20260809 07:51:24.476788 16360 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:24.476835 16356 heartbeater.cc:344] Connected to a master server at 127.26.37.60:43251
I20260809 07:51:24.476987 16356 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:24.478057 16360 heartbeater.cc:507] Master 127.26.37.61:37063 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:24.479425 16356 heartbeater.cc:507] Master 127.26.37.60:43251 requested a full tablet report, sending...
I20260809 07:51:24.480818 15870 ts_manager.cc:194] Registered new tserver with Master: b44794667c9d4a8f9cab88bf9746a495 (127.26.37.2:41717)
I20260809 07:51:24.481814 16359 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:24.481958 16359 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:24.482470 16359 heartbeater.cc:507] Master 127.26.37.62:40109 requested a full tablet report, sending...
I20260809 07:51:24.483563 15609 ts_manager.cc:194] Registered new tserver with Master: b44794667c9d4a8f9cab88bf9746a495 (127.26.37.2:41717)
I20260809 07:51:24.483815 15745 ts_manager.cc:194] Registered new tserver with Master: b44794667c9d4a8f9cab88bf9746a495 (127.26.37.2:41717)
I20260809 07:51:24.484543 15609 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:51311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:24.564977 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:24.889855 16171 heartbeater.cc:499] Master 127.26.37.62:40109 was elected leader, sending a full tablet report...
W20260809 07:51:24.922709 16377 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:24.923059 16377 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:24.923213 16377 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:24.931968 16377 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:24.932201 16377 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:51:24.942845 16377 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:24.944475 16377 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:24.945901 16377 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:24.955449 16384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:24.955819 16382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:24.956526 16386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:24.957113 16377 server_base.cc:1061] running on GCE node
I20260809 07:51:24.957764 16377 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:24.957955 16377 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:24.958055 16377 hybrid_clock.cc:648] HybridClock initialized: now 1786261884958031 us; error 0 us; skew 500 ppm
I20260809 07:51:24.958379 16377 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:24 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:24.962208 16377 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:51:24.963840 16377 webserver.cc:533] Webserver started at http://127.26.37.3:34063/ using document root <none> and password file <none>
I20260809 07:51:24.964588 16377 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:24.964743 16377 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:24.970296 16377 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:51:24.973026 16393 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:24.974432 16377 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:24.974632 16377 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "8fee1a9e44fe49769b45ebf615d0c8b5"
format_stamp: "Formatted at 2026-08-09 07:51:15 on dist-test-slave-bw88"
I20260809 07:51:24.975354 16377 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:25.006847 16377 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:25.011195 16377 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:25.011579 16377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:25.012643 16377 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:25.014340 16377 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:25.014468 16377 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.014614 16377 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:25.014715 16377 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.051812 16377 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:35823
I20260809 07:51:25.051870 16556 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:35823 every 8 connection(s)
I20260809 07:51:25.053519 16377 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:25.062688 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16377
I20260809 07:51:25.062915 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:51:25 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:25 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:25.112061 16559 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37063
Aug 09 07:51:25 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:25.112571 16559 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:25.113788 16559 heartbeater.cc:507] Master 127.26.37.61:37063 requested a full tablet report, sending...
I20260809 07:51:25.115903 15745 ts_manager.cc:194] Registered new tserver with Master: 8fee1a9e44fe49769b45ebf615d0c8b5 (127.26.37.3:35823)
I20260809 07:51:25.118932 16558 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:25.119127 16558 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:25.119764 16558 heartbeater.cc:507] Master 127.26.37.62:40109 requested a full tablet report, sending...
I20260809 07:51:25.120870 15609 ts_manager.cc:194] Registered new tserver with Master: 8fee1a9e44fe49769b45ebf615d0c8b5 (127.26.37.3:35823)
I20260809 07:51:25.121858 15609 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:43603
I20260809 07:51:25.122417 16557 heartbeater.cc:344] Connected to a master server at 127.26.37.60:43251
I20260809 07:51:25.122582 16557 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:25.123160 16557 heartbeater.cc:507] Master 127.26.37.60:43251 requested a full tablet report, sending...
I20260809 07:51:25.125267 15870 ts_manager.cc:194] Registered new tserver with Master: 8fee1a9e44fe49769b45ebf615d0c8b5 (127.26.37.3:35823)
I20260809 07:51:25.139137 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:25.147688 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:25.147904 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15565
I20260809 07:51:25.148801 15651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:25.258983 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15565
I20260809 07:51:25.287417 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40109
--webserver_interface=127.26.37.62
--webserver_port=37349
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:25.487082 16359 heartbeater.cc:499] Master 127.26.37.62:40109 was elected leader, sending a full tablet report...
W20260809 07:51:25.488611 16359 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:40109 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:40109: connect: Connection refused (error 111)
W20260809 07:51:25.641633 16579 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:25.641999 16579 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:25.642160 16579 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:25.650151 16579 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:25.650285 16579 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:25.650414 16579 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:25.650503 16579 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:25.660714 16579 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=37349
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:25.662710 16579 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:25.664374 16579 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:25.673477 16588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:25.673816 16589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:25.674687 16593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:25.675822 16579 server_base.cc:1061] running on GCE node
I20260809 07:51:25.676537 16579 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:25.676738 16579 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:25.676834 16579 hybrid_clock.cc:648] HybridClock initialized: now 1786261885676793 us; error 0 us; skew 500 ppm
I20260809 07:51:25.677184 16579 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:25 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261885, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:25.681097 16579 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:25.683106 16579 webserver.cc:533] Webserver started at http://127.26.37.62:37349/ using document root <none> and password file <none>
I20260809 07:51:25.683923 16579 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:25.684077 16579 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:25.689390 16579 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:25.692142 16599 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.693578 16579 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:51:25.693755 16579 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "a533c201e39c434899abc9487d1d5c8c"
format_stamp: "Formatted at 2026-08-09 07:51:12 on dist-test-slave-bw88"
I20260809 07:51:25.694458 16579 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:25.712105 16579 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:25.716187 16579 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:25.716464 16579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:25.737118 16579 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:40109
I20260809 07:51:25.737136 16676 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:40109 every 8 connection(s)
I20260809 07:51:25.738744 16579 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:25.743336 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16579
I20260809 07:51:25.743695 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15969
I20260809 07:51:25.746284 16678 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:25.750434 16678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap starting.
Aug 09 07:51:25 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:51:25.763540 16161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:25.771584 16678 log.cc:826] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:25.780648 16678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:25.781247 16678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: Bootstrap complete.
I20260809 07:51:25.788303 16678 raft_consensus.cc:359] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:25.789041 16678 raft_consensus.cc:740] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a533c201e39c434899abc9487d1d5c8c, State: Initialized, Role: FOLLOWER
I20260809 07:51:25.789920 16678 consensus_queue.cc:260] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:25.790789 16688 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:25.791085 16688 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:25.791757 16678 sys_catalog.cc:565] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:25.802781 16701 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:25.802959 16701 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: loading cluster ID for follower catalog manager: success
I20260809 07:51:25.807360 16701 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: acquiring CA information for follower catalog manager: success
I20260809 07:51:25.810047 16701 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:25.850725 16579 master_runner.cc:264] No leader master found from master a533c201e39c434899abc9487d1d5c8c
I20260809 07:51:25.850884 16579 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:25.863934 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15969
I20260809 07:51:25.888931 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46563
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--webserver_port=39163
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:26.124931 16558 heartbeater.cc:499] Master 127.26.37.62:40109 was elected leader, sending a full tablet report...
I20260809 07:51:26.139007 16629 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8fee1a9e44fe49769b45ebf615d0c8b5" instance_seqno: 1786261885042628) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:54955; Asking this server to re-register.
I20260809 07:51:26.139977 16558 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:26.141812 16629 ts_manager.cc:194] Registered new tserver with Master: 8fee1a9e44fe49769b45ebf615d0c8b5 (127.26.37.3:35823)
W20260809 07:51:26.242717 16707 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:26.243046 16707 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:26.243208 16707 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:26.251106 16707 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:26.251335 16707 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:26.261018 16707 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39163
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:26.262871 16707 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:26.264227 16707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:26.273644 16718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:26.273674 16716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:26.274078 16721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:26.274860 16707 server_base.cc:1061] running on GCE node
I20260809 07:51:26.275344 16707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:26.275478 16707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:26.275554 16707 hybrid_clock.cc:648] HybridClock initialized: now 1786261886275537 us; error 0 us; skew 500 ppm
I20260809 07:51:26.275799 16707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:26 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261886, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:26.279462 16707 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:26.281294 16707 webserver.cc:533] Webserver started at http://127.26.37.1:39163/ using document root <none> and password file <none>
I20260809 07:51:26.282203 16707 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:26.282352 16707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:26.287669 16707 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 07:51:26.290329 16729 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:26.291712 16707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:26.291885 16707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:26.292613 16707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:26.309934 16707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:26.313652 16707 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:26.313934 16707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:26.314863 16707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:26.316493 16707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:26.316620 16707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:26.316761 16707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:26.316851 16707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:26.352861 16707 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46563
I20260809 07:51:26.352945 16885 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46563 every 8 connection(s)
I20260809 07:51:26.354437 16707 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:26.358875 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16707
Aug 09 07:51:26 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261886, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:26 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261886, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:26 dist-test-slave-bw88 krb5kdc[15546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261886, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:26.395172 16886 heartbeater.cc:344] Connected to a master server at 127.26.37.60:43251
I20260809 07:51:26.395195 16889 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:26.395658 16890 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37063
I20260809 07:51:26.395656 16889 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:26.395830 16890 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:26.396715 16890 heartbeater.cc:507] Master 127.26.37.61:37063 requested a full tablet report, sending...
I20260809 07:51:26.396739 16889 heartbeater.cc:507] Master 127.26.37.62:40109 requested a full tablet report, sending...
I20260809 07:51:26.395658 16886 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:26.397457 16886 heartbeater.cc:507] Master 127.26.37.60:43251 requested a full tablet report, sending...
I20260809 07:51:26.398979 16629 ts_manager.cc:194] Registered new tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:46563)
I20260809 07:51:26.399184 15870 ts_manager.cc:194] Re-registered known tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:46563)
I20260809 07:51:26.399365 15745 ts_manager.cc:194] Re-registered known tserver with Master: ff53db6fe82b4dc39921be5827c0472f (127.26.37.1:46563)
I20260809 07:51:26.414253 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16707
W20260809 07:51:26.415464 16904 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:26.411907 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.26.37.1:46563)
0809 07:51:26.412575 (+ 668us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:26.412582 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:26.412613 (+ 31us) server_negotiation.cc:408] Connection header received
0809 07:51:26.412667 (+ 54us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:26.412674 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:26.412880 (+ 206us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:26.413043 (+ 163us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:26.413964 (+ 921us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48896: BlockingRecv error: recv got EOF from 127.0.0.1:48896 (error 108)
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":340,"threads_started":1}
W20260809 07:51:26.416831 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16707: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:46563: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:26.417011 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16707
I20260809 07:51:26.444442 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46563
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--webserver_port=39163
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:26.495810 16359 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40109
I20260809 07:51:26.496053 16359 heartbeater.cc:499] Master 127.26.37.62:40109 was elected leader, sending a full tablet report...
I20260809 07:51:26.496960 16629 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b44794667c9d4a8f9cab88bf9746a495" instance_seqno: 1786261884417482) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:51207; Asking this server to re-register.
I20260809 07:51:26.497572 16359 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:26.498786 16629 ts_manager.cc:194] Registered new tserver with Master: b44794667c9d4a8f9cab88bf9746a495 (127.26.37.2:41717)
I20260809 07:51:26.519889 16909 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a533c201e39c434899abc9487d1d5c8c)
I20260809 07:51:26.520105 16909 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:26.522037 16909 leader_election.cc:290] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a533c201e39c434899abc9487d1d5c8c (127.26.37.62:40109), 376a94640d2d4012bdcfadec09302ab3 (127.26.37.61:37063)
I20260809 07:51:26.522894 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ba678862f634122ab88fea23f1e7932" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "376a94640d2d4012bdcfadec09302ab3" is_pre_election: true
I20260809 07:51:26.529250 16643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ba678862f634122ab88fea23f1e7932" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a533c201e39c434899abc9487d1d5c8c" is_pre_election: true
I20260809 07:51:26.529790 16643 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba678862f634122ab88fea23f1e7932 in term 2.
I20260809 07:51:26.530385 15848 leader_election.cc:304] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ba678862f634122ab88fea23f1e7932, a533c201e39c434899abc9487d1d5c8c; no voters: 376a94640d2d4012bdcfadec09302ab3
I20260809 07:51:26.530803 16909 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:51:26.530965 16909 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a533c201e39c434899abc9487d1d5c8c)
I20260809 07:51:26.531090 16909 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:26.534804 16909 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:26.535727 16909 leader_election.cc:290] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [CANDIDATE]: Term 3 election: Requested vote from peers a533c201e39c434899abc9487d1d5c8c (127.26.37.62:40109), 376a94640d2d4012bdcfadec09302ab3 (127.26.37.61:37063)
I20260809 07:51:26.536196 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ba678862f634122ab88fea23f1e7932" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "376a94640d2d4012bdcfadec09302ab3"
I20260809 07:51:26.536167 16643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ba678862f634122ab88fea23f1e7932" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a533c201e39c434899abc9487d1d5c8c"
I20260809 07:51:26.536427 16643 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:26.541864 16643 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba678862f634122ab88fea23f1e7932 in term 3.
I20260809 07:51:26.542366 15848 leader_election.cc:304] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ba678862f634122ab88fea23f1e7932, a533c201e39c434899abc9487d1d5c8c; no voters: 376a94640d2d4012bdcfadec09302ab3
I20260809 07:51:26.542701 16909 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:51:26.543203 16909 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [term 3 LEADER]: Becoming Leader. State: Replica: 6ba678862f634122ab88fea23f1e7932, State: Running, Role: LEADER
I20260809 07:51:26.543637 16909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:26.545501 16913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ba678862f634122ab88fea23f1e7932. Latest consensus state: current_term: 3 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:26.545849 16913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:26.546583 16915 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:51:26.548458 16915 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:51:26.549466 16915 catalog_manager.cc:1295] Loaded cluster ID: d43f5c2518d54efdbdfedeb10e553dc1
I20260809 07:51:26.549568 16915 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:51:26.550717 16915 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:51:26.551527 16915 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932: Loaded TSK: 0
I20260809 07:51:26.552301 16915 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:51:26.600625 16919 raft_consensus.cc:493] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a533c201e39c434899abc9487d1d5c8c)
I20260809 07:51:26.600883 16919 raft_consensus.cc:515] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } }
I20260809 07:51:26.602953 16919 leader_election.cc:290] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a533c201e39c434899abc9487d1d5c8c (127.26.37.62:40109), 6ba678862f634122ab88fea23f1e7932 (127.26.37.60:43251)
I20260809 07:51:26.612591 16643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "376a94640d2d4012bdcfadec09302ab3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a533c201e39c434899abc9487d1d5c8c" is_pre_election: true
I20260809 07:51:26.612937 15886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "376a94640d2d4012bdcfadec09302ab3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ba678862f634122ab88fea23f1e7932" is_pre_election: true
I20260809 07:51:26.613037 16643 raft_consensus.cc:2393] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 376a94640d2d4012bdcfadec09302ab3 in current term 3: Already voted for candidate 6ba678862f634122ab88fea23f1e7932 in this term.
I20260809 07:51:26.613929 15719 leader_election.cc:304] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 376a94640d2d4012bdcfadec09302ab3; no voters: 6ba678862f634122ab88fea23f1e7932, a533c201e39c434899abc9487d1d5c8c
I20260809 07:51:26.614414 16919 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:26.616683 16919 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 07:51:26.823195 16906 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:26.823559 16906 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:26.823724 16906 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:51:26.823815 16906 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:26.831892 16906 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:26.832165 16906 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:26.842159 16906 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39163
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:26.843765 16906 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:26.845085 16906 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:26.855037 16931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:26.855104 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:26.855177 16930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:26.856177 16906 server_base.cc:1061] running on GCE node
I20260809 07:51:26.856667 16906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:26.856784 16906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:26.856854 16906 hybrid_clock.cc:648] HybridClock initialized: now 1786261886856827 us; error 0 us; skew 500 ppm
I20260809 07:51:26.857081 16906 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:26 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261886, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:26.860738 16906 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:26.862347 16906 webserver.cc:533] Webserver started at http://127.26.37.1:39163/ using document root <none> and password file <none>
I20260809 07:51:26.863059 16906 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:26.863147 16906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:26.868274 16906 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:51:26.871197 16944 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:26.872529 16906 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:51:26.872674 16906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:26.873417 16906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:26.895869 16906 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:26.903990 16906 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:26.904325 16906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:26.905387 16906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:26.907022 16906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:26.907127 16906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:26.907296 16906 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:26.907362 16906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:51:26.907768 16906 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:51:26.907830 16906 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:51:26.907912 16906 fault_injection.cc:59] injecting fault for kudu (pid 16906): FLAGS_fault_before_start (process will exit)
I20260809 07:51:26.931856 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46563
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--webserver_port=39163
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:26.982599 16643 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [term 3 FOLLOWER]: Refusing update from remote peer 6ba678862f634122ab88fea23f1e7932: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:51:26.983541 16913 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:51:26.987344 16955 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ba678862f634122ab88fea23f1e7932. Latest consensus state: current_term: 3 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:26.987596 16955 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:26.989892 16909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:26.990181 16909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:26.990898 16955 sys_catalog.cc:455] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:26.990998 15758 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [term 3 FOLLOWER]: Refusing update from remote peer 6ba678862f634122ab88fea23f1e7932: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:51:26.991080 16955 sys_catalog.cc:458] T 00000000000000000000000000000000 P a533c201e39c434899abc9487d1d5c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:26.991668 16909 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:51:26.994305 16919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ba678862f634122ab88fea23f1e7932. Latest consensus state: current_term: 3 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:26.994261 16966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:26.994570 16919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:26.994613 16966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376a94640d2d4012bdcfadec09302ab3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:51:27.290042 16954 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:27.290328 16954 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:27.290500 16954 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:27.298765 16954 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:27.298995 16954 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:27.309535 16954 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39163
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40109,127.26.37.61:37063,127.26.37.60:43251
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:27.311172 16954 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:27.312534 16954 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:27.322499 16974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:27.322657 16976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:27.322697 16973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:27.323910 16954 server_base.cc:1061] running on GCE node
I20260809 07:51:27.324455 16954 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:51:27.324653 16954 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:51:27.324753 16954 hybrid_clock.cc:648] HybridClock initialized: now 1786261887324717 us; error 0 us; skew 500 ppm
I20260809 07:51:27.325055 16954 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:27 dist-test-slave-bw88 krb5kdc[15546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261887, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:27.328845 16954 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:27.330651 16954 webserver.cc:533] Webserver started at http://127.26.37.1:39163/ using document root <none> and password file <none>
I20260809 07:51:27.331348 16954 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:27.331442 16954 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:27.336503 16954 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:51:27.339149 16984 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:27.340556 16954 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:51:27.340727 16954 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ff53db6fe82b4dc39921be5827c0472f"
format_stamp: "Formatted at 2026-08-09 07:51:14 on dist-test-slave-bw88"
I20260809 07:51:27.341544 16954 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:27.357506 16954 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:27.360879 16954 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:27.361147 16954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:27.362191 16954 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:27.363829 16954 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:27.363924 16954 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:27.364070 16954 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:27.364167 16954 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:27.398593 16954 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46563
I20260809 07:51:27.398756 17151 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46563 every 8 connection(s)
I20260809 07:51:27.400430 16954 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:27.408464 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16954
I20260809 07:51:27.408890 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16954
W20260809 07:51:27.434856 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16954: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:46563: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:27.435070 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16954
I20260809 07:51:27.463169 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16188
W20260809 07:51:27.464112 17160 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:27.405123 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:45533 (local address 127.26.37.61:37063)
0809 07:51:27.415005 (+ 9882us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:27.415015 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:27.427071 (+ 12056us) server_negotiation.cc:408] Connection header received
0809 07:51:27.427347 (+ 276us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:27.427354 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:27.427436 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:27.427523 (+ 87us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:27.432205 (+ 4682us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:27.433690 (+ 1485us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:27.462983 (+ 29293us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:45533: BlockingRecv error: recv got EOF from 127.26.37.1:45533 (error 108)
Metrics: {"server-negotiator.queue_time_us":9719,"thread_start_us":9526,"threads_started":1}
W20260809 07:51:27.464118 17157 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:27.404171 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:57709 (local address 127.26.37.62:40109)
0809 07:51:27.419950 (+ 15779us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:27.419959 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:27.425974 (+ 6015us) server_negotiation.cc:408] Connection header received
0809 07:51:27.426152 (+ 178us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:27.426158 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:27.426245 (+ 87us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:27.426367 (+ 122us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:27.430025 (+ 3658us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:27.431244 (+ 1219us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:27.433842 (+ 2598us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:27.434400 (+ 558us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:51:27.463097 (+ 28697us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:57709: BlockingRecv error: recv got EOF from 127.26.37.1:57709 (error 108)
Metrics: {"server-negotiator.queue_time_us":15623,"thread_start_us":17205,"threads_started":1}
W20260809 07:51:27.464118 17164 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:27.405541 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:57455 (local address 127.26.37.60:43251)
0809 07:51:27.425187 (+ 19646us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:27.425196 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:27.425222 (+ 26us) server_negotiation.cc:408] Connection header received
0809 07:51:27.425303 (+ 81us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:27.425312 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:27.425449 (+ 137us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:27.425595 (+ 146us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:27.429419 (+ 3824us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:27.431960 (+ 2541us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:27.434602 (+ 2642us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:27.435118 (+ 516us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:51:27.463097 (+ 27979us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:57455: BlockingRecv error: recv got EOF from 127.26.37.1:57455 (error 108)
Metrics: {"server-negotiator.queue_time_us":19490,"thread_start_us":19332,"threads_started":1}
W20260809 07:51:27.466799 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16188: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:41717: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:27.466943 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16188
I20260809 07:51:27.490482 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16377
W20260809 07:51:27.493275 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16377: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:35823: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:27.493481 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16377
I20260809 07:51:27.520097 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16579
W20260809 07:51:27.523293 17157 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:27.521370 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.26.37.62:40109)
0809 07:51:27.521558 (+ 188us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:27.521563 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:27.521580 (+ 17us) server_negotiation.cc:408] Connection header received
0809 07:51:27.521815 (+ 235us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:27.521822 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:27.521914 (+ 92us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:27.521997 (+ 83us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:27.523043 (+ 1046us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49888: BlockingRecv error: recv got EOF from 127.0.0.1:49888 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 07:51:27.523480 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16579: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:40109: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:27.523660 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16579
I20260809 07:51:27.531997 16909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6ba678862f634122ab88fea23f1e7932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a533c201e39c434899abc9487d1d5c8c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40109 } } peers { permanent_uuid: "376a94640d2d4012bdcfadec09302ab3" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37063 } } peers { permanent_uuid: "6ba678862f634122ab88fea23f1e7932" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 43251 } } }
I20260809 07:51:27.532335 16909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba678862f634122ab88fea23f1e7932 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:27.552609 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15694
I20260809 07:51:27.553720 15788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:27.670219 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15694
I20260809 07:51:27.700662 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15826
I20260809 07:51:27.701930 15915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:27.811439 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15826
[ OK ] ExternalMiniClusterTest.TestBasicOperation/6 (18529 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/7
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:51:27 dist-test-slave-bw88 krb5kdc[17180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:27 dist-test-slave-bw88 krb5kdc[17180](info): set up 2 sockets
Aug 09 07:51:27 dist-test-slave-bw88 krb5kdc[17180](info): commencing operation
krb5kdc: starting...
W20260809 07:51:30.002664 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.150s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:51:30 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:51:30Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:30.323086 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45345
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:30.670450 17209 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:30.670800 17209 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:30.670967 17209 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:30.678982 17209 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:30.679070 17209 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:30.679174 17209 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:30.679240 17209 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:30.689641 17209 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:30.691239 17209 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:30.692679 17209 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:30.700999 17216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:30.700995 17217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:30.701785 17220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:30.702836 17209 server_base.cc:1061] running on GCE node
I20260809 07:51:30.703540 17209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:30.704895 17209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:30.706091 17209 hybrid_clock.cc:648] HybridClock initialized: now 1786261890706052 us; error 35 us; skew 500 ppm
I20260809 07:51:30.706418 17209 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:30 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:30.709632 17209 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:30.711222 17209 webserver.cc:533] Webserver started at http://127.26.37.62:36129/ using document root <none> and password file <none>
I20260809 07:51:30.711928 17209 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:30.712030 17209 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:30.712334 17209 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:30.714792 17209 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "7ba91afbe1cc4dcf9372df3276a56035"
format_stamp: "Formatted at 2026-08-09 07:51:30 on dist-test-slave-bw88"
I20260809 07:51:30.715579 17209 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "7ba91afbe1cc4dcf9372df3276a56035"
format_stamp: "Formatted at 2026-08-09 07:51:30 on dist-test-slave-bw88"
I20260809 07:51:30.721261 17209 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 07:51:30.724834 17230 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:30.726363 17209 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:30.726585 17209 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "7ba91afbe1cc4dcf9372df3276a56035"
format_stamp: "Formatted at 2026-08-09 07:51:30 on dist-test-slave-bw88"
I20260809 07:51:30.726830 17209 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:30.748178 17209 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:30.752022 17209 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:30.752318 17209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:30.773797 17209 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:45345
I20260809 07:51:30.773901 17305 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:45345 every 8 connection(s)
I20260809 07:51:30.775332 17209 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:30.779855 17306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:30.780417 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 17209
I20260809 07:51:30.780665 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:51:30.786162 17306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:30 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
I20260809 07:51:30.810774 17306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:30.812438 17237 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37019: connect: Connection refused (error 111)
W20260809 07:51:30.814327 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37019: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37019: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
I20260809 07:51:30.863722 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } attempt: 1
W20260809 07:51:30.866084 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37019: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37019: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:30.896972 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37019
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:30.934405 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } attempt: 2
W20260809 07:51:30.936474 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37019: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37019: connect: Connection refused (error 111)
I20260809 07:51:31.027796 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } attempt: 3
W20260809 07:51:31.030092 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37019: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37019: connect: Connection refused (error 111)
I20260809 07:51:31.173441 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } attempt: 4
W20260809 07:51:31.175832 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37019: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37019: connect: Connection refused (error 111)
W20260809 07:51:31.246224 17318 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:31.246555 17318 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:31.246721 17318 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:31.254964 17318 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:31.255046 17318 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:31.255167 17318 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:31.255239 17318 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:31.269891 17318 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:31.272210 17318 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:31.274274 17318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:31.282466 17329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:31.282603 17328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:31.284170 17331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:31.284453 17318 server_base.cc:1061] running on GCE node
I20260809 07:51:31.285233 17318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:31.286612 17318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:31.287851 17318 hybrid_clock.cc:648] HybridClock initialized: now 1786261891287810 us; error 55 us; skew 500 ppm
I20260809 07:51:31.288208 17318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:31.291422 17318 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:51:31.293048 17318 webserver.cc:533] Webserver started at http://127.26.37.61:41525/ using document root <none> and password file <none>
I20260809 07:51:31.293851 17318 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:31.294009 17318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:31.294356 17318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:31.296708 17318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "8c99e566e0f84663bf9e5e6f211a06bc"
format_stamp: "Formatted at 2026-08-09 07:51:31 on dist-test-slave-bw88"
I20260809 07:51:31.297719 17318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "8c99e566e0f84663bf9e5e6f211a06bc"
format_stamp: "Formatted at 2026-08-09 07:51:31 on dist-test-slave-bw88"
I20260809 07:51:31.303208 17318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:51:31.306667 17340 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:31.308032 17318 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:31.308224 17318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "8c99e566e0f84663bf9e5e6f211a06bc"
format_stamp: "Formatted at 2026-08-09 07:51:31 on dist-test-slave-bw88"
I20260809 07:51:31.308439 17318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:31.333767 17318 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:31.337538 17318 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:31.337836 17318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:31.358876 17318 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:37019
I20260809 07:51:31.358945 17407 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:37019 every 8 connection(s)
I20260809 07:51:31.360478 17318 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:51:31.363670 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 17318
I20260809 07:51:31.363909 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:51:31.365453 17409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:31.374432 17409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
I20260809 07:51:31.396453 17409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:31.413704 17409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:31.414867 17344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
W20260809 07:51:31.416792 17409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:51:31.466199 17409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 1
W20260809 07:51:31.468531 17409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
I20260809 07:51:31.475106 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } attempt: 5
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:31.487021 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:33215
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:31.490927 17306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:31.492875 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
I20260809 07:51:31.536801 17409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 2
W20260809 07:51:31.539834 17409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
I20260809 07:51:31.544124 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 1
W20260809 07:51:31.546214 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
I20260809 07:51:31.614476 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 2
W20260809 07:51:31.616844 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
I20260809 07:51:31.631085 17409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 3
W20260809 07:51:31.633456 17409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
I20260809 07:51:31.723117 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 3
W20260809 07:51:31.725286 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
I20260809 07:51:31.776726 17409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 4
W20260809 07:51:31.779091 17409 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
W20260809 07:51:31.835515 17427 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:31.835829 17427 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:31.836009 17427 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:31.844180 17427 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:31.844264 17427 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:31.844354 17427 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:31.844426 17427 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:31.854511 17427 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:33215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:31.856115 17427 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:31.857482 17427 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:31.865710 17432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:31.865621 17433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:31.866971 17438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:31.867828 17427 server_base.cc:1061] running on GCE node
I20260809 07:51:31.868573 17427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:31.869983 17427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:31.871212 17427 hybrid_clock.cc:648] HybridClock initialized: now 1786261891871169 us; error 50 us; skew 500 ppm
I20260809 07:51:31.871558 17427 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:31.874882 17427 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:51:31.876425 17427 webserver.cc:533] Webserver started at http://127.26.37.60:40719/ using document root <none> and password file <none>
I20260809 07:51:31.877140 17427 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:31.877281 17427 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:31.877642 17427 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:31.879961 17427 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "cd429eb1e5e24c54a0009ed345e3e291"
format_stamp: "Formatted at 2026-08-09 07:51:31 on dist-test-slave-bw88"
I20260809 07:51:31.880748 17427 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "cd429eb1e5e24c54a0009ed345e3e291"
format_stamp: "Formatted at 2026-08-09 07:51:31 on dist-test-slave-bw88"
I20260809 07:51:31.886454 17427 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 07:51:31.889763 17449 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:31.891037 17427 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:31.891230 17427 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "cd429eb1e5e24c54a0009ed345e3e291"
format_stamp: "Formatted at 2026-08-09 07:51:31 on dist-test-slave-bw88"
I20260809 07:51:31.891467 17427 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:31.902534 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 4
W20260809 07:51:31.904371 17306 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111)
I20260809 07:51:31.910338 17427 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:31.919130 17427 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:31.919431 17427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:31.940462 17427 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:33215
I20260809 07:51:31.940503 17522 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:33215 every 8 connection(s)
I20260809 07:51:31.942200 17427 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:51:31.943612 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 17427
I20260809 07:51:31.943807 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:51:31.947721 17523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:31.954296 17523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:31.976095 17523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:31.991283 17523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:31 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:32.009166 17523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Bootstrap starting.
I20260809 07:51:32.012382 17523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:32.013576 17523 log.cc:826] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:32.016244 17523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: No bootstrap required, opened a new log
I20260809 07:51:32.021090 17523 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:32.021477 17523 raft_consensus.cc:385] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:32.021605 17523 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd429eb1e5e24c54a0009ed345e3e291, State: Initialized, Role: FOLLOWER
I20260809 07:51:32.022190 17523 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:32.023428 17534 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.023778 17534 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:32.024509 17523 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:51:32.033856 17550 catalog_manager.cc:1594] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:32.033979 17550 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:32.031158 17427 master_runner.cc:187] Error getting master registration for 127.26.37.62:45345: OK, instance_id { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" instance_seqno: 1786261890768134 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:51:32.038715 17427 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:32.078336 17409 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 5
Aug 09 07:51:32 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261891, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:32.096751 17409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Bootstrap starting.
I20260809 07:51:32.100617 17409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:32.101893 17409 log.cc:826] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:32.104410 17409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: No bootstrap required, opened a new log
I20260809 07:51:32.109535 17409 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:32.109840 17409 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:32.110015 17409 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c99e566e0f84663bf9e5e6f211a06bc, State: Initialized, Role: FOLLOWER
I20260809 07:51:32.110663 17409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:32.111686 17553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.112076 17553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:32.113080 17409 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:51:32.124315 17568 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:32.124512 17568 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:32.121390 17318 master_runner.cc:187] Error getting master registration for 127.26.37.62:45345: OK, instance_id { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" instance_seqno: 1786261890768134 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:51:32.128361 17318 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:51:32.171711 17462 tablet.cc:2406] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:32.181607 17306 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } attempt: 5
Aug 09 07:51:32 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:32.204977 17306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap starting.
I20260809 07:51:32.209195 17306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:32.210736 17306 log.cc:826] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:32.216086 17306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: No bootstrap required, opened a new log
I20260809 07:51:32.223855 17306 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:32.224177 17306 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:32.224305 17306 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba91afbe1cc4dcf9372df3276a56035, State: Initialized, Role: FOLLOWER
I20260809 07:51:32.225057 17306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:32.226208 17572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.226583 17572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:32.227408 17306 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:32.237012 17534 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:32.237251 17534 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:32.239014 17534 leader_election.cc:290] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ba91afbe1cc4dcf9372df3276a56035 (127.26.37.62:45345), 8c99e566e0f84663bf9e5e6f211a06bc (127.26.37.61:37019)
I20260809 07:51:32.239642 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd429eb1e5e24c54a0009ed345e3e291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba91afbe1cc4dcf9372df3276a56035" is_pre_election: true
I20260809 07:51:32.240226 17265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd429eb1e5e24c54a0009ed345e3e291 in term 0.
I20260809 07:51:32.240402 17375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd429eb1e5e24c54a0009ed345e3e291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" is_pre_election: true
I20260809 07:51:32.240895 17458 leader_election.cc:304] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba91afbe1cc4dcf9372df3276a56035, cd429eb1e5e24c54a0009ed345e3e291; no voters:
I20260809 07:51:32.240947 17375 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd429eb1e5e24c54a0009ed345e3e291 in term 0.
I20260809 07:51:32.241544 17534 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:51:32.241703 17534 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:32.241818 17534 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:32.245297 17534 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:32.246331 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd429eb1e5e24c54a0009ed345e3e291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba91afbe1cc4dcf9372df3276a56035"
I20260809 07:51:32.246522 17265 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:32.250274 17265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd429eb1e5e24c54a0009ed345e3e291 in term 1.
I20260809 07:51:32.250825 17458 leader_election.cc:304] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba91afbe1cc4dcf9372df3276a56035, cd429eb1e5e24c54a0009ed345e3e291; no voters:
I20260809 07:51:32.254204 17375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd429eb1e5e24c54a0009ed345e3e291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c99e566e0f84663bf9e5e6f211a06bc"
I20260809 07:51:32.254302 17534 leader_election.cc:290] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [CANDIDATE]: Term 1 election: Requested vote from peers 7ba91afbe1cc4dcf9372df3276a56035 (127.26.37.62:45345), 8c99e566e0f84663bf9e5e6f211a06bc (127.26.37.61:37019)
I20260809 07:51:32.254472 17375 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:32.254555 17534 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:32.254683 17534 raft_consensus.cc:697] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 1 LEADER]: Becoming Leader. State: Replica: cd429eb1e5e24c54a0009ed345e3e291, State: Running, Role: LEADER
I20260809 07:51:32.255172 17534 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
W20260809 07:51:32.255601 17240 tablet.cc:2406] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:32.258255 17375 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd429eb1e5e24c54a0009ed345e3e291 in term 1.
I20260809 07:51:32.276134 17582 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd429eb1e5e24c54a0009ed345e3e291. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.276583 17582 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:32.284626 17590 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:51:32.284853 17589 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:51:32.287485 17589 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:51:32.304893 17375 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 1 FOLLOWER]: Refusing update from remote peer cd429eb1e5e24c54a0009ed345e3e291: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:32.305159 17265 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Refusing update from remote peer cd429eb1e5e24c54a0009ed345e3e291: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:32.306185 17582 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:32.306891 17534 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:32.330258 17572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd429eb1e5e24c54a0009ed345e3e291. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.330544 17572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:32.335810 17593 mvcc.cc:204] Tried to move back new op lower bound from 7316528710871031808 to 7316528710708752384. Current Snapshot: MvccSnapshot[applied={T|T < 7316528710871031808}]
I20260809 07:51:32.337961 17594 mvcc.cc:204] Tried to move back new op lower bound from 7316528710871031808 to 7316528710708752384. Current Snapshot: MvccSnapshot[applied={T|T < 7316528710871031808}]
W20260809 07:51:32.340687 17346 tablet.cc:2406] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:32.341789 17582 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.342007 17582 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:32.344570 17553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd429eb1e5e24c54a0009ed345e3e291. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.344849 17553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:32.345022 17589 catalog_manager.cc:1383] Generated new cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:32.345135 17589 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:51:32.350670 17534 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.350888 17534 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:32.354288 17572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.354467 17572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:32.365962 17553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:32.366153 17553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:32.390255 17589 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:51:32.392318 17589 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:51:32.424388 17589 catalog_manager.cc:6092] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Generated new TSK 0
I20260809 07:51:32.425740 17589 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:32.588939 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:32.980023 17612 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:32.980410 17612 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:32.980590 17612 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:32.989183 17612 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:32.989432 17612 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:33.000185 17612 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:33.002295 17612 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:33.003726 17612 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:33.014901 17623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:33.015062 17628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:33.014875 17624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:33.016139 17612 server_base.cc:1061] running on GCE node
I20260809 07:51:33.016815 17612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:33.017783 17612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:33.018975 17612 hybrid_clock.cc:648] HybridClock initialized: now 1786261893018934 us; error 36 us; skew 500 ppm
I20260809 07:51:33.019250 17612 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:33 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:33.022576 17612 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:33.024274 17612 webserver.cc:533] Webserver started at http://127.26.37.1:45625/ using document root <none> and password file <none>
I20260809 07:51:33.025022 17612 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:33.025156 17612 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:33.025583 17612 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:33.028147 17612 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:33.029042 17612 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:33.034756 17612 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 07:51:33.038172 17638 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:33.039492 17612 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:33.039674 17612 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:33.039873 17612 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:33.062721 17612 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:33.066522 17612 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:33.066849 17612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:33.067819 17612 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:33.069420 17612 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:33.069491 17612 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:33.069664 17612 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:33.069725 17612 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:33.112406 17612 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:33965
I20260809 07:51:33.112450 17803 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:33965 every 8 connection(s)
I20260809 07:51:33.113962 17612 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:33.123349 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 17612
I20260809 07:51:33.123660 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:51:33.127372 17568 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:33.127511 17568 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: loading cluster ID for follower catalog manager: success
I20260809 07:51:33.134056 17568 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: acquiring CA information for follower catalog manager: success
I20260809 07:51:33.142051 17568 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:51:33 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:33 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:33 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:33.159382 17807 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:33.159837 17807 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:33.161042 17807 heartbeater.cc:507] Master 127.26.37.62:45345 requested a full tablet report, sending...
I20260809 07:51:33.162782 17812 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37019
I20260809 07:51:33.162956 17812 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:33.163410 17812 heartbeater.cc:507] Master 127.26.37.61:37019 requested a full tablet report, sending...
I20260809 07:51:33.163874 17253 ts_manager.cc:194] Registered new tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:33965)
I20260809 07:51:33.164925 17804 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33215
I20260809 07:51:33.165066 17804 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:33.165283 17361 ts_manager.cc:194] Registered new tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:33965)
I20260809 07:51:33.165567 17804 heartbeater.cc:507] Master 127.26.37.60:33215 requested a full tablet report, sending...
I20260809 07:51:33.167075 17480 ts_manager.cc:194] Registered new tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:33965)
I20260809 07:51:33.168507 17480 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:53879
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:33.257247 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:33.290477 17590 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:33.290611 17590 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: loading cluster ID for follower catalog manager: success
I20260809 07:51:33.294301 17590 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: acquiring CA information for follower catalog manager: success
I20260809 07:51:33.296715 17590 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:51:33.610778 17826 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:33.611148 17826 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:33.611305 17826 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:33.620162 17826 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:33.620406 17826 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:51:33.631192 17826 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:33.632815 17826 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:33.634336 17826 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:33.643903 17834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:33.644582 17836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:33.645073 17838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:33.646179 17826 server_base.cc:1061] running on GCE node
I20260809 07:51:33.646713 17826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:33.647646 17826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:33.648885 17826 hybrid_clock.cc:648] HybridClock initialized: now 1786261893648824 us; error 43 us; skew 500 ppm
I20260809 07:51:33.649192 17826 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:33 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:33.652545 17826 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:51:33.654327 17826 webserver.cc:533] Webserver started at http://127.26.37.2:44761/ using document root <none> and password file <none>
I20260809 07:51:33.655043 17826 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:33.655176 17826 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:33.655532 17826 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:33.658356 17826 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "25e0f3d3c31d4d218833ac36648d4a07"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:33.659250 17826 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "25e0f3d3c31d4d218833ac36648d4a07"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:33.664849 17826 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:51:33.668272 17849 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:33.669731 17826 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:33.669894 17826 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "25e0f3d3c31d4d218833ac36648d4a07"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:33.670120 17826 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:33.688583 17826 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:33.692919 17826 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:33.693269 17826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:33.694284 17826 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:33.695919 17826 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:33.695990 17826 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:33.696151 17826 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:33.696203 17826 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:33.733803 17826 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:38949
I20260809 07:51:33.733888 18013 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:38949 every 8 connection(s)
I20260809 07:51:33.735327 17826 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:33.742388 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 17826
I20260809 07:51:33.742627 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:51:33 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:33 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:33 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261893, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
I20260809 07:51:33.785833 18016 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33215
I20260809 07:51:33.786293 18016 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:33.787494 18016 heartbeater.cc:507] Master 127.26.37.60:33215 requested a full tablet report, sending...
I20260809 07:51:33.788170 18017 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:33.788298 18017 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:33.788820 18017 heartbeater.cc:507] Master 127.26.37.62:45345 requested a full tablet report, sending...
I20260809 07:51:33.789431 17480 ts_manager.cc:194] Registered new tserver with Master: 25e0f3d3c31d4d218833ac36648d4a07 (127.26.37.2:38949)
I20260809 07:51:33.789837 17253 ts_manager.cc:194] Registered new tserver with Master: 25e0f3d3c31d4d218833ac36648d4a07 (127.26.37.2:38949)
I20260809 07:51:33.790356 18018 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37019
I20260809 07:51:33.790356 17480 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:37225
I20260809 07:51:33.790532 18018 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:33.791167 18018 heartbeater.cc:507] Master 127.26.37.61:37019 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.3@KRBTEST.COM" created.
I20260809 07:51:33.793416 17361 ts_manager.cc:194] Registered new tserver with Master: 25e0f3d3c31d4d218833ac36648d4a07 (127.26.37.2:38949)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:33.875348 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:34.170853 17804 heartbeater.cc:499] Master 127.26.37.60:33215 was elected leader, sending a full tablet report...
W20260809 07:51:34.218660 18035 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:34.218952 18035 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:34.219050 18035 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:34.227098 18035 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:34.227311 18035 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:51:34.237686 18035 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:34.239123 18035 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:34.240299 18035 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:34.250204 18046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:34.250259 18041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:34.251289 18049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:34.251750 18035 server_base.cc:1061] running on GCE node
I20260809 07:51:34.252290 18035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:34.253273 18035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:34.254523 18035 hybrid_clock.cc:648] HybridClock initialized: now 1786261894254478 us; error 40 us; skew 500 ppm
I20260809 07:51:34.254851 18035 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:34 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:34.258153 18035 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:51:34.259826 18035 webserver.cc:533] Webserver started at http://127.26.37.3:35881/ using document root <none> and password file <none>
I20260809 07:51:34.260524 18035 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:34.260656 18035 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:34.260972 18035 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:34.263486 18035 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "e1f3b1764de34c8ea92fc79d53991336"
format_stamp: "Formatted at 2026-08-09 07:51:34 on dist-test-slave-bw88"
I20260809 07:51:34.264302 18035 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "e1f3b1764de34c8ea92fc79d53991336"
format_stamp: "Formatted at 2026-08-09 07:51:34 on dist-test-slave-bw88"
I20260809 07:51:34.269711 18035 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 07:51:34.272848 18056 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:34.274223 18035 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:34.274400 18035 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "e1f3b1764de34c8ea92fc79d53991336"
format_stamp: "Formatted at 2026-08-09 07:51:34 on dist-test-slave-bw88"
I20260809 07:51:34.274601 18035 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:34.291083 18035 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:34.294838 18035 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:34.295128 18035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:34.296078 18035 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:34.297845 18035 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:34.297946 18035 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:34.298089 18035 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:34.298161 18035 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:34.334470 18035 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:40059
I20260809 07:51:34.334533 18216 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:40059 every 8 connection(s)
I20260809 07:51:34.335837 18035 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:34.340848 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18035
I20260809 07:51:34.341109 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:51:34 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:34.378643 18217 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33215
Aug 09 07:51:34 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:34.379081 18217 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:34.380263 18217 heartbeater.cc:507] Master 127.26.37.60:33215 requested a full tablet report, sending...
Aug 09 07:51:34 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:34.382299 17480 ts_manager.cc:194] Registered new tserver with Master: e1f3b1764de34c8ea92fc79d53991336 (127.26.37.3:40059)
I20260809 07:51:34.383534 17480 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:43829
I20260809 07:51:34.388729 18218 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:34.388844 18219 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37019
I20260809 07:51:34.388911 18218 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:34.389048 18219 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:34.389420 18218 heartbeater.cc:507] Master 127.26.37.62:45345 requested a full tablet report, sending...
I20260809 07:51:34.389456 18219 heartbeater.cc:507] Master 127.26.37.61:37019 requested a full tablet report, sending...
I20260809 07:51:34.390545 17253 ts_manager.cc:194] Registered new tserver with Master: e1f3b1764de34c8ea92fc79d53991336 (127.26.37.3:40059)
I20260809 07:51:34.390583 17361 ts_manager.cc:194] Registered new tserver with Master: e1f3b1764de34c8ea92fc79d53991336 (127.26.37.3:40059)
I20260809 07:51:34.396318 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:34.403462 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:34.403650 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17209
I20260809 07:51:34.404525 17299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:34.518375 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 17209
I20260809 07:51:34.551708 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45345
--webserver_interface=127.26.37.62
--webserver_port=36129
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:34.792846 18016 heartbeater.cc:499] Master 127.26.37.60:33215 was elected leader, sending a full tablet report...
W20260809 07:51:34.794857 18017 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:45345 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:45345: connect: Connection refused (error 111)
W20260809 07:51:34.916530 18229 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:34.916927 18229 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:34.917035 18229 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:34.925671 18229 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:34.925787 18229 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:34.925848 18229 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:34.925904 18229 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:34.936894 18229 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=36129
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:34.938644 18229 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:34.940251 18229 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:34.942241 17458 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:45345: connect: Connection refused (error 111)
W20260809 07:51:34.946686 17458 consensus_peers.cc:597] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 -> Peer 7ba91afbe1cc4dcf9372df3276a56035 (127.26.37.62:45345): Couldn't send request to peer 7ba91afbe1cc4dcf9372df3276a56035. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:45345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:51:34.950649 18241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:34.951406 18243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:34.951948 18246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:34.954059 18229 server_base.cc:1061] running on GCE node
I20260809 07:51:34.954802 18229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:34.956246 18229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:34.957435 18229 hybrid_clock.cc:648] HybridClock initialized: now 1786261894957389 us; error 67 us; skew 500 ppm
I20260809 07:51:34.957794 18229 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:34 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:34.961294 18229 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:34.963080 18229 webserver.cc:533] Webserver started at http://127.26.37.62:36129/ using document root <none> and password file <none>
I20260809 07:51:34.963835 18229 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:34.963997 18229 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:34.969554 18229 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:34.972227 18259 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:34.973584 18229 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:34.973765 18229 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "7ba91afbe1cc4dcf9372df3276a56035"
format_stamp: "Formatted at 2026-08-09 07:51:30 on dist-test-slave-bw88"
I20260809 07:51:34.974500 18229 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:34.986007 18229 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:34.989559 18229 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:34.989821 18229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:35.010330 18229 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:45345
I20260809 07:51:35.010342 18327 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:45345 every 8 connection(s)
I20260809 07:51:35.011968 18229 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:35.019130 18328 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:35.019513 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18229
I20260809 07:51:35.019790 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17612
I20260809 07:51:35.023324 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap starting.
Aug 09 07:51:35 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:51:35.040704 17797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:35.043704 18328 log.cc:826] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:35.054855 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:35.055541 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap complete.
I20260809 07:51:35.065434 18328 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:35.066336 18328 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba91afbe1cc4dcf9372df3276a56035, State: Initialized, Role: FOLLOWER
I20260809 07:51:35.067129 18328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:35.067902 18337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:35.068152 18337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:35.069013 18328 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:35.108907 18354 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:35.109102 18354 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: loading cluster ID for follower catalog manager: success
I20260809 07:51:35.114467 18354 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: acquiring CA information for follower catalog manager: success
Aug 09 07:51:35 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261894, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:35.117048 18354 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:35.164216 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 17612
I20260809 07:51:35.194780 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:33965
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--webserver_port=45625
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:35.386242 18217 heartbeater.cc:499] Master 127.26.37.60:33215 was elected leader, sending a full tablet report...
I20260809 07:51:35.403599 18280 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e1f3b1764de34c8ea92fc79d53991336" instance_seqno: 1786261894325682) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:35375; Asking this server to re-register.
I20260809 07:51:35.404610 18218 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:35.406597 18280 ts_manager.cc:194] Registered new tserver with Master: e1f3b1764de34c8ea92fc79d53991336 (127.26.37.3:40059)
I20260809 07:51:35.461870 17534 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:35.462180 17534 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:35.496992 18337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd429eb1e5e24c54a0009ed345e3e291. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:35.497320 18337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:51:35.563643 18362 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:35.564116 18362 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:35.564263 18362 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:35.573439 18362 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:35.573670 18362 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:35.584798 18362 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:33965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45625
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:35.586608 18362 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:35.588052 18362 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:35.598742 18376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:35.598776 18375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:35.598806 18379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:35.599715 18362 server_base.cc:1061] running on GCE node
I20260809 07:51:35.600270 18362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:35.601305 18362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:35.602589 18362 hybrid_clock.cc:648] HybridClock initialized: now 1786261895602510 us; error 66 us; skew 500 ppm
I20260809 07:51:35.602993 18362 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:35 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261895, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:35.606714 18362 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:35.608739 18362 webserver.cc:533] Webserver started at http://127.26.37.1:45625/ using document root <none> and password file <none>
I20260809 07:51:35.609678 18362 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:35.609819 18362 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:35.616096 18362 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:51:35.619185 18388 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:35.620738 18362 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:51:35.620939 18362 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:35.621767 18362 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:35.644605 18362 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:35.649315 18362 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:35.649729 18362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:35.650916 18362 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:35.652693 18362 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:35.652832 18362 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:35.652969 18362 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:35.653064 18362 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:35.691565 18362 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:33965
I20260809 07:51:35.691658 18540 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:33965 every 8 connection(s)
I20260809 07:51:35.693219 18362 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:35.693773 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18362
Aug 09 07:51:35 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261895, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:35 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261895, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:35 dist-test-slave-bw88 krb5kdc[17180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261895, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:35.725997 18542 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:35.726418 18542 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:35.727579 18542 heartbeater.cc:507] Master 127.26.37.62:45345 requested a full tablet report, sending...
I20260809 07:51:35.729871 18280 ts_manager.cc:194] Registered new tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:33965)
I20260809 07:51:35.732951 18541 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33215
I20260809 07:51:35.733121 18541 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:35.733763 18541 heartbeater.cc:507] Master 127.26.37.60:33215 requested a full tablet report, sending...
I20260809 07:51:35.735101 17480 ts_manager.cc:194] Re-registered known tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:33965)
I20260809 07:51:35.735828 18543 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37019
I20260809 07:51:35.735953 18543 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:35.736253 17480 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:35451
I20260809 07:51:35.736378 18543 heartbeater.cc:507] Master 127.26.37.61:37019 requested a full tablet report, sending...
I20260809 07:51:35.737902 17361 ts_manager.cc:194] Re-registered known tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:33965)
I20260809 07:51:35.761085 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18362
W20260809 07:51:35.762564 18556 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:35.758954 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.26.37.1:33965)
0809 07:51:35.759606 (+ 652us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:35.759613 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:35.759644 (+ 31us) server_negotiation.cc:408] Connection header received
0809 07:51:35.759703 (+ 59us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:35.759709 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:35.759917 (+ 208us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:35.760097 (+ 180us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:35.760939 (+ 842us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33792: BlockingRecv error: recv got EOF from 127.0.0.1:33792 (error 108)
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":347,"threads_started":1}
W20260809 07:51:35.764215 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18362: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:33965: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:35.764386 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18362
I20260809 07:51:35.791857 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:33965
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--webserver_port=45625
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:35.802615 18017 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:35.803786 18280 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "25e0f3d3c31d4d218833ac36648d4a07" instance_seqno: 1786261893724366) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:47349; Asking this server to re-register.
I20260809 07:51:35.804409 18017 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:35.805653 18280 ts_manager.cc:194] Registered new tserver with Master: 25e0f3d3c31d4d218833ac36648d4a07 (127.26.37.2:38949)
I20260809 07:51:36.068965 18366 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd429eb1e5e24c54a0009ed345e3e291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:36.069236 18366 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: LEADER
W20260809 07:51:36.146762 18561 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:36.147094 18561 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:36.147187 18561 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:51:36.147248 18561 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:36.156364 18561 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:36.156610 18561 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:36.168213 18561 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:33965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45625
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:36.170063 18561 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:36.171657 18561 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:36.182462 18576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:36.182505 18575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:36.183836 18580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:36.184589 18561 server_base.cc:1061] running on GCE node
I20260809 07:51:36.185228 18561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:36.186379 18561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:36.187677 18561 hybrid_clock.cc:648] HybridClock initialized: now 1786261896187615 us; error 67 us; skew 500 ppm
I20260809 07:51:36.188015 18561 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:36 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261896, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:36.191922 18561 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:36.193806 18561 webserver.cc:533] Webserver started at http://127.26.37.1:45625/ using document root <none> and password file <none>
I20260809 07:51:36.194657 18561 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:36.194796 18561 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:36.200937 18561 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 07:51:36.204103 18589 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:36.205557 18561 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:51:36.205758 18561 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:36.206561 18561 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:36.224504 18561 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:36.228789 18561 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:36.229125 18561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:36.230253 18561 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:36.231981 18561 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:36.232086 18561 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:36.232245 18561 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:36.232316 18561 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:51:36.232744 18561 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:51:36.232813 18561 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:51:36.232897 18561 fault_injection.cc:59] injecting fault for kudu (pid 18561): FLAGS_fault_before_start (process will exit)
I20260809 07:51:36.262557 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:33965
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--webserver_port=45625
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:36.623880 18601 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:36.624202 18601 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:36.624295 18601 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:36.632735 18601 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:36.632937 18601 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:36.643726 18601 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:33965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45625
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:36.645334 18601 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:36.646744 18601 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:36.656744 18609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:36.656733 18608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:36.657375 18613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:36.657791 18601 server_base.cc:1061] running on GCE node
I20260809 07:51:36.658380 18601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:36.659266 18601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:36.660495 18601 hybrid_clock.cc:648] HybridClock initialized: now 1786261896660451 us; error 53 us; skew 500 ppm
I20260809 07:51:36.660807 18601 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:36 dist-test-slave-bw88 krb5kdc[17180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261896, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:36.664238 18601 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:36.665923 18601 webserver.cc:533] Webserver started at http://127.26.37.1:45625/ using document root <none> and password file <none>
I20260809 07:51:36.666605 18601 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:36.666705 18601 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:36.672063 18601 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:51:36.674793 18622 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:36.676158 18601 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:36.676337 18601 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:36.677063 18601 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:36.699412 18601 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:36.703666 18601 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:36.703979 18601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:36.705030 18601 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:36.706651 18601 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:36.706734 18601 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:36.706902 18601 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:36.706996 18601 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:36.742066 18601 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:33965
I20260809 07:51:36.742247 18777 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:33965 every 8 connection(s)
I20260809 07:51:36.743624 18601 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:36.752725 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18601
I20260809 07:51:36.753201 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18601
W20260809 07:51:36.763784 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18601: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:33965: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:36.763974 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18601
I20260809 07:51:36.791787 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17826
W20260809 07:51:36.792456 18784 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:36.746996 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:36407 (local address 127.26.37.60:33215)
0809 07:51:36.758187 (+ 11191us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:36.758196 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:36.763663 (+ 5467us) server_negotiation.cc:408] Connection header received
0809 07:51:36.791487 (+ 27824us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:36407: BlockingRecv error: recv got EOF from 127.26.37.1:36407 (error 108)
Metrics: {"server-negotiator.queue_time_us":11033,"thread_start_us":11191,"threads_started":1}
W20260809 07:51:36.792464 18798 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:36.764593 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:52887 (local address 127.26.37.61:37019)
0809 07:51:36.766338 (+ 1745us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:36.766343 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:36.791515 (+ 25172us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:52887: BlockingRecv error: recv got EOF from 127.26.37.1:52887 (error 108)
Metrics: {"server-negotiator.queue_time_us":1651,"thread_start_us":7773,"threads_started":1}
W20260809 07:51:36.792481 18790 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:36.748387 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:44315 (local address 127.26.37.62:45345)
0809 07:51:36.761768 (+ 13381us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:36.761775 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:36.762603 (+ 828us) server_negotiation.cc:408] Connection header received
0809 07:51:36.762680 (+ 77us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:36.762687 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:36.762776 (+ 89us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:36.762888 (+ 112us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:36.791549 (+ 28661us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:44315: BlockingRecv error: recv got EOF from 127.26.37.1:44315 (error 108)
Metrics: {"server-negotiator.queue_time_us":13239,"thread_start_us":13146,"threads_started":1}
W20260809 07:51:36.794904 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17826: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:38949: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:36.795070 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 17826
I20260809 07:51:36.820539 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18035
W20260809 07:51:36.823568 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18035: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:40059: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:36.823689 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18035
I20260809 07:51:36.849978 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18229
W20260809 07:51:36.852286 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18229: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:45345: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:51:36.852412 18790 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:36.850879 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:59104 (local address 127.26.37.62:45345)
0809 07:51:36.851075 (+ 196us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:36.851083 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:36.851104 (+ 21us) server_negotiation.cc:408] Connection header received
0809 07:51:36.851338 (+ 234us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:36.851344 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:36.851421 (+ 77us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:36.851528 (+ 107us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:36.852214 (+ 686us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59104: BlockingRecv error: recv got EOF from 127.0.0.1:59104 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 07:51:36.852473 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18229
I20260809 07:51:36.879148 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17318
I20260809 07:51:36.880579 17401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:36.991659 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 17318
I20260809 07:51:37.018076 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17427
I20260809 07:51:37.019342 17516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:37.134775 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 17427
2026-08-09T07:51:37Z chronyd exiting
Aug 09 07:51:37 dist-test-slave-bw88 krb5kdc[18808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:37 dist-test-slave-bw88 krb5kdc[18808](info): set up 2 sockets
Aug 09 07:51:37 dist-test-slave-bw88 krb5kdc[18808](info): commencing operation
krb5kdc: starting...
W20260809 07:51:39.280586 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.109s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:51:39 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:51:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:39.557015 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45345
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:39.899590 18831 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:39.899950 18831 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:39.900116 18831 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:39.908159 18831 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:39.908257 18831 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:39.908344 18831 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:39.908432 18831 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:39.919044 18831 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:39.920671 18831 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:39.922130 18831 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:39.930403 18842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:39.930452 18840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:39.931221 18845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:39.931734 18831 server_base.cc:1061] running on GCE node
I20260809 07:51:39.932504 18831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:39.933964 18831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:39.935204 18831 hybrid_clock.cc:648] HybridClock initialized: now 1786261899935168 us; error 59 us; skew 500 ppm
I20260809 07:51:39.935534 18831 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:39 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261899, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:39.939055 18831 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:39.940572 18831 webserver.cc:533] Webserver started at http://127.26.37.62:33963/ using document root <none> and password file <none>
I20260809 07:51:39.941301 18831 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:39.941469 18831 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:39.946805 18831 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:51:39.949647 18857 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:39.951009 18831 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:39.951138 18831 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "7ba91afbe1cc4dcf9372df3276a56035"
format_stamp: "Formatted at 2026-08-09 07:51:30 on dist-test-slave-bw88"
I20260809 07:51:39.951882 18831 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:39.965970 18831 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:39.975208 18831 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:39.975440 18831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:39.995805 18831 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:45345
I20260809 07:51:39.995834 18932 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:45345 every 8 connection(s)
I20260809 07:51:39.997484 18831 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:40.002367 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18831
I20260809 07:51:40.002604 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:51:40.004365 18933 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:40.007670 18933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap starting.
I20260809 07:51:40.024649 18933 log.cc:826] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:40.033690 18933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:40.034219 18933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap complete.
I20260809 07:51:40.040828 18933 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:40.041599 18933 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba91afbe1cc4dcf9372df3276a56035, State: Initialized, Role: FOLLOWER
I20260809 07:51:40.042303 18933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:40.043198 18942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:40.043488 18942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:40.044293 18933 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:40.055333 18958 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:40.055464 18958 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: loading cluster ID for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:40.059991 18958 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: acquiring CA information for follower catalog manager: success
I20260809 07:51:40.062554 18958 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:51:40 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261899, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:40.081924 18831 master_runner.cc:187] Error getting master registration for 127.26.37.61:37019: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37019: connect: Connection refused (error 111),
I20260809 07:51:40.087639 18831 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:40.118383 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37019
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:40.471540 18967 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:40.471984 18967 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:40.472167 18967 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:40.480870 18967 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:40.480983 18967 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:40.481098 18967 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:40.481222 18967 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:40.492457 18967 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:40.494266 18967 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:40.495852 18967 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:40.504626 18976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:40.504664 18975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:40.505570 18981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:40.506414 18967 server_base.cc:1061] running on GCE node
I20260809 07:51:40.507215 18967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:40.508638 18967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:40.509867 18967 hybrid_clock.cc:648] HybridClock initialized: now 1786261900509806 us; error 47 us; skew 500 ppm
I20260809 07:51:40.510248 18967 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:40 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261900, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:40.513815 18967 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:51:40.515735 18967 webserver.cc:533] Webserver started at http://127.26.37.61:34765/ using document root <none> and password file <none>
I20260809 07:51:40.516758 18967 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:40.516909 18967 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:40.522631 18967 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:51:40.525556 18990 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:40.526916 18967 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:40.527128 18967 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "8c99e566e0f84663bf9e5e6f211a06bc"
format_stamp: "Formatted at 2026-08-09 07:51:31 on dist-test-slave-bw88"
I20260809 07:51:40.527837 18967 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:40.546237 18967 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:40.550474 18967 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:40.550746 18967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:40.572327 18967 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:37019
I20260809 07:51:40.572346 19064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:37019 every 8 connection(s)
I20260809 07:51:40.574143 18967 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:51:40.576376 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18967
I20260809 07:51:40.576602 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:51:40.582032 19065 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:40.587003 19065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Bootstrap starting.
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:40.610004 19065 log.cc:826] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:40.619287 19065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:40.619880 19065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Bootstrap complete.
I20260809 07:51:40.631029 19065 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:40.632143 19065 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c99e566e0f84663bf9e5e6f211a06bc, State: Initialized, Role: FOLLOWER
I20260809 07:51:40.633170 19065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:40.634162 19070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:40.634442 19070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:40.635697 19065 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:40.647531 19083 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:40.647749 19083 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: loading cluster ID for follower catalog manager: success
I20260809 07:51:40.653436 19083 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: acquiring CA information for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:40.656810 19083 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:40.686911 18967 master_runner.cc:187] Error getting master registration for 127.26.37.60:33215: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33215: connect: Connection refused (error 111),
I20260809 07:51:40.692410 18967 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:40.718627 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:33215
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:41.075129 19092 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:41.075531 19092 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:41.075682 19092 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:41.083940 19092 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:41.084090 19092 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:41.084208 19092 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:41.084290 19092 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:41.097064 19092 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:33215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:41.098896 19092 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:41.100265 19092 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:41.108894 19104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:41.108923 19102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:41.109561 19106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:41.110234 19092 server_base.cc:1061] running on GCE node
I20260809 07:51:41.110883 19092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:41.112201 19092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:41.113436 19092 hybrid_clock.cc:648] HybridClock initialized: now 1786261901113364 us; error 59 us; skew 500 ppm
I20260809 07:51:41.113747 19092 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:41 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261901, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:41.117300 19092 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:51:41.118935 19092 webserver.cc:533] Webserver started at http://127.26.37.60:33625/ using document root <none> and password file <none>
I20260809 07:51:41.119680 19092 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:41.119830 19092 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:41.125464 19092 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:51:41.128268 19116 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:41.129675 19092 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:51:41.129860 19092 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "cd429eb1e5e24c54a0009ed345e3e291"
format_stamp: "Formatted at 2026-08-09 07:51:31 on dist-test-slave-bw88"
I20260809 07:51:41.130582 19092 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:41.144938 19092 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:41.149012 19092 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:41.149281 19092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:41.170004 19092 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:33215
I20260809 07:51:41.170022 19182 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:33215 every 8 connection(s)
I20260809 07:51:41.171629 19092 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:51:41.176405 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19092
I20260809 07:51:41.176627 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:51:41.179558 19184 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:41.184152 19184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Bootstrap starting.
Aug 09 07:51:41 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:41.203591 19184 log.cc:826] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:51:41 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:41.212960 19184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:41.213590 19184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: Bootstrap complete.
I20260809 07:51:41.219949 19184 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:41.220597 19184 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd429eb1e5e24c54a0009ed345e3e291, State: Initialized, Role: FOLLOWER
Aug 09 07:51:41 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:41.221325 19184 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:41.222317 19193 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:41.222707 19193 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:41.223518 19184 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:41.235280 19208 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:41.235378 19208 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: loading cluster ID for follower catalog manager: success
I20260809 07:51:41.239010 19208 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: acquiring CA information for follower catalog manager: success
Aug 09 07:51:41 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261901, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:41.241681 19208 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:41.269418 19092 master_runner.cc:264] No leader master found from master cd429eb1e5e24c54a0009ed345e3e291
I20260809 07:51:41.269562 19092 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:41.365881 19213 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:41.366171 19213 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:41.368146 19213 leader_election.cc:290] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c99e566e0f84663bf9e5e6f211a06bc (127.26.37.61:37019), cd429eb1e5e24c54a0009ed345e3e291 (127.26.37.60:33215)
I20260809 07:51:41.376189 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ba91afbe1cc4dcf9372df3276a56035" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" is_pre_election: true
I20260809 07:51:41.376732 19027 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba91afbe1cc4dcf9372df3276a56035 in term 1.
I20260809 07:51:41.376870 19151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ba91afbe1cc4dcf9372df3276a56035" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd429eb1e5e24c54a0009ed345e3e291" is_pre_election: true
I20260809 07:51:41.377405 18862 leader_election.cc:304] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba91afbe1cc4dcf9372df3276a56035, 8c99e566e0f84663bf9e5e6f211a06bc; no voters:
I20260809 07:51:41.377552 19151 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba91afbe1cc4dcf9372df3276a56035 in term 1.
I20260809 07:51:41.377810 19213 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:51:41.377954 19213 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:41.378063 19213 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:51:41.383507 19213 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:41.384289 19213 leader_election.cc:290] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [CANDIDATE]: Term 2 election: Requested vote from peers 8c99e566e0f84663bf9e5e6f211a06bc (127.26.37.61:37019), cd429eb1e5e24c54a0009ed345e3e291 (127.26.37.60:33215)
I20260809 07:51:41.384709 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ba91afbe1cc4dcf9372df3276a56035" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8c99e566e0f84663bf9e5e6f211a06bc"
I20260809 07:51:41.384902 19027 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:51:41.385149 19151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ba91afbe1cc4dcf9372df3276a56035" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd429eb1e5e24c54a0009ed345e3e291"
I20260809 07:51:41.385370 19151 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:51:41.389257 19151 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba91afbe1cc4dcf9372df3276a56035 in term 2.
I20260809 07:51:41.389803 18862 leader_election.cc:304] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba91afbe1cc4dcf9372df3276a56035, cd429eb1e5e24c54a0009ed345e3e291; no voters:
I20260809 07:51:41.390141 19213 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:51:41.390149 19027 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba91afbe1cc4dcf9372df3276a56035 in term 2.
I20260809 07:51:41.390805 19213 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 2 LEADER]: Becoming Leader. State: Replica: 7ba91afbe1cc4dcf9372df3276a56035, State: Running, Role: LEADER
I20260809 07:51:41.391283 19213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:41.393602 19220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba91afbe1cc4dcf9372df3276a56035. Latest consensus state: current_term: 2 leader_uuid: "7ba91afbe1cc4dcf9372df3276a56035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:41.394239 19220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:41.394977 19222 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:51:41.396894 19222 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:51:41.397758 19222 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:41.397861 19222 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:51:41.398942 19222 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:51:41.399727 19222 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Loaded TSK: 0
I20260809 07:51:41.400565 19222 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:41.549099 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:41.864157 19027 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 2 FOLLOWER]: Refusing update from remote peer 7ba91afbe1cc4dcf9372df3276a56035: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:51:41.865250 19220 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:51:41.869494 19232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba91afbe1cc4dcf9372df3276a56035. Latest consensus state: current_term: 2 leader_uuid: "7ba91afbe1cc4dcf9372df3276a56035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:41.869875 19232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:41.870800 19220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "7ba91afbe1cc4dcf9372df3276a56035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:41.871088 19220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:41.872824 19151 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 2 FOLLOWER]: Refusing update from remote peer 7ba91afbe1cc4dcf9372df3276a56035: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:51:41.873975 19220 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:51:41.889833 19232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "7ba91afbe1cc4dcf9372df3276a56035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:41.890069 19232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:41.890718 19241 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "7ba91afbe1cc4dcf9372df3276a56035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:41.891086 19241 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:41.891749 19237 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba91afbe1cc4dcf9372df3276a56035. Latest consensus state: current_term: 2 leader_uuid: "7ba91afbe1cc4dcf9372df3276a56035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:41.892019 19237 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:51:41.913877 19230 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:41.914153 19230 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:41.914254 19230 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:41.921958 19230 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:41.922118 19230 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:41.932096 19230 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:41.933673 19230 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:41.934964 19230 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:41.944451 19249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:41.944480 19248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:41.945955 19252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:41.946516 19230 server_base.cc:1061] running on GCE node
I20260809 07:51:41.947108 19230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:41.948101 19230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:41.949376 19230 hybrid_clock.cc:648] HybridClock initialized: now 1786261901949268 us; error 56 us; skew 500 ppm
I20260809 07:51:41.949689 19230 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:41 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261901, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:41.953235 19230 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:41.954869 19230 webserver.cc:533] Webserver started at http://127.26.37.1:41781/ using document root <none> and password file <none>
I20260809 07:51:41.955605 19230 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:41.955750 19230 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:41.961021 19230 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:41.963726 19267 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:41.965026 19230 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:51:41.965219 19230 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:41.965991 19230 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:42.000223 19230 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:42.005137 19230 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:42.005436 19230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:42.006544 19230 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:42.008399 19230 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:42.008512 19230 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:42.008605 19230 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:42.008679 19230 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:42.043776 19230 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44493
I20260809 07:51:42.043888 19435 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44493 every 8 connection(s)
I20260809 07:51:42.045423 19230 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:42.050063 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19230
I20260809 07:51:42.050302 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:51:42 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261901, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:42 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261901, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:42 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261901, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:42.089664 19440 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37019
I20260809 07:51:42.090138 19440 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:42.090790 19438 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:42.090932 19438 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:42.091320 19440 heartbeater.cc:507] Master 127.26.37.61:37019 requested a full tablet report, sending...
I20260809 07:51:42.091531 19438 heartbeater.cc:507] Master 127.26.37.62:45345 requested a full tablet report, sending...
I20260809 07:51:42.094199 19013 ts_manager.cc:194] Registered new tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:44493)
I20260809 07:51:42.095738 18880 ts_manager.cc:194] Registered new tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:44493)
I20260809 07:51:42.098021 18880 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:33759
I20260809 07:51:42.098711 19437 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33215
I20260809 07:51:42.098860 19437 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:42.099319 19437 heartbeater.cc:507] Master 127.26.37.60:33215 requested a full tablet report, sending...
I20260809 07:51:42.100772 19139 ts_manager.cc:194] Registered new tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:44493)
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:42.198601 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:42.493818 19213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "7ba91afbe1cc4dcf9372df3276a56035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:42.494274 19213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: LEADER
W20260809 07:51:42.546502 19458 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:42.546831 19458 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:42.546988 19458 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:42.555500 19458 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:42.555769 19458 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:51:42.571776 19458 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:42.573993 19458 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:42.575765 19458 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:42.587033 19470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:42.587009 19469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:42.589957 19473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:42.590618 19458 server_base.cc:1061] running on GCE node
I20260809 07:51:42.591414 19458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:42.592908 19458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:42.594141 19458 hybrid_clock.cc:648] HybridClock initialized: now 1786261902594060 us; error 57 us; skew 500 ppm
I20260809 07:51:42.594494 19458 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:42 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261902, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:42.600136 19458 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:51:42.602555 19458 webserver.cc:533] Webserver started at http://127.26.37.2:43363/ using document root <none> and password file <none>
I20260809 07:51:42.603642 19458 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:42.603813 19458 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:42.611770 19458 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 07:51:42.615549 19483 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:42.617328 19458 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 07:51:42.617585 19458 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "25e0f3d3c31d4d218833ac36648d4a07"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:42.618556 19458 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:42.650738 19458 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:42.656351 19458 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:42.656682 19458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:42.657959 19458 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:42.660028 19458 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:42.660138 19458 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:42.660251 19458 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:42.660349 19458 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:42.694137 19458 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:37715
I20260809 07:51:42.694240 19642 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:37715 every 8 connection(s)
I20260809 07:51:42.695782 19458 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:42.698138 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19458
I20260809 07:51:42.698398 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:51:42 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261902, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Aug 09 07:51:42 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261902, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:42 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261902, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:42.748546 19645 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33215
I20260809 07:51:42.748948 19645 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:42.750671 19645 heartbeater.cc:507] Master 127.26.37.60:33215 requested a full tablet report, sending...
I20260809 07:51:42.752876 19139 ts_manager.cc:194] Registered new tserver with Master: 25e0f3d3c31d4d218833ac36648d4a07 (127.26.37.2:37715)
I20260809 07:51:42.756026 19646 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:42.756152 19646 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:42.756577 19652 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37019
I20260809 07:51:42.756735 19646 heartbeater.cc:507] Master 127.26.37.62:45345 requested a full tablet report, sending...
I20260809 07:51:42.756734 19652 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:42.757325 19652 heartbeater.cc:507] Master 127.26.37.61:37019 requested a full tablet report, sending...
I20260809 07:51:42.757902 18880 ts_manager.cc:194] Registered new tserver with Master: 25e0f3d3c31d4d218833ac36648d4a07 (127.26.37.2:37715)
I20260809 07:51:42.758504 19013 ts_manager.cc:194] Registered new tserver with Master: 25e0f3d3c31d4d218833ac36648d4a07 (127.26.37.2:37715)
I20260809 07:51:42.759001 18880 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:49045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:42.828537 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:43.101181 19438 heartbeater.cc:499] Master 127.26.37.62:45345 was elected leader, sending a full tablet report...
W20260809 07:51:43.182190 19671 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:43.182549 19671 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:43.182720 19671 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:43.190539 19671 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:43.190761 19671 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:51:43.201270 19671 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:43.202939 19671 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:43.204403 19671 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:43.214931 19679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:43.215155 19681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:43.215104 19677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:43.216629 19671 server_base.cc:1061] running on GCE node
I20260809 07:51:43.217222 19671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:43.218209 19671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:43.219452 19671 hybrid_clock.cc:648] HybridClock initialized: now 1786261903219398 us; error 67 us; skew 500 ppm
I20260809 07:51:43.219802 19671 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:43 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261903, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:43.223624 19671 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:51:43.225383 19671 webserver.cc:533] Webserver started at http://127.26.37.3:35919/ using document root <none> and password file <none>
I20260809 07:51:43.226191 19671 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:43.226343 19671 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:43.231841 19671 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 07:51:43.234769 19697 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:43.236229 19671 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:43.236399 19671 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "e1f3b1764de34c8ea92fc79d53991336"
format_stamp: "Formatted at 2026-08-09 07:51:34 on dist-test-slave-bw88"
I20260809 07:51:43.237138 19671 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:43.251529 19671 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:43.255546 19671 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:43.255818 19671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:43.256906 19671 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:43.258473 19671 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:43.258605 19671 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:43.258729 19671 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:43.258833 19671 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:43.295133 19671 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:35657
I20260809 07:51:43.295244 19858 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:35657 every 8 connection(s)
I20260809 07:51:43.296748 19671 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:43.304369 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19671
I20260809 07:51:43.304634 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:51:43 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261903, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:43 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261903, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:43 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261903, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:43.339756 19859 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33215
I20260809 07:51:43.340148 19859 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:43.341235 19859 heartbeater.cc:507] Master 127.26.37.60:33215 requested a full tablet report, sending...
I20260809 07:51:43.342252 19860 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:43.342397 19860 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:43.342917 19860 heartbeater.cc:507] Master 127.26.37.62:45345 requested a full tablet report, sending...
I20260809 07:51:43.343181 19139 ts_manager.cc:194] Registered new tserver with Master: e1f3b1764de34c8ea92fc79d53991336 (127.26.37.3:35657)
I20260809 07:51:43.343933 18879 ts_manager.cc:194] Registered new tserver with Master: e1f3b1764de34c8ea92fc79d53991336 (127.26.37.3:35657)
I20260809 07:51:43.345130 18879 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:48711
I20260809 07:51:43.345551 19862 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37019
I20260809 07:51:43.345741 19862 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:43.346247 19862 heartbeater.cc:507] Master 127.26.37.61:37019 requested a full tablet report, sending...
I20260809 07:51:43.347289 19013 ts_manager.cc:194] Registered new tserver with Master: e1f3b1764de34c8ea92fc79d53991336 (127.26.37.3:35657)
I20260809 07:51:43.356160 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:43.362708 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:43.362893 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18831
I20260809 07:51:43.363754 18927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:43.471131 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18831
I20260809 07:51:43.498051 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45345
--webserver_interface=127.26.37.62
--webserver_port=33963
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:43.761739 19646 heartbeater.cc:499] Master 127.26.37.62:45345 was elected leader, sending a full tablet report...
W20260809 07:51:43.763298 19646 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:45345 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:45345: connect: Connection refused (error 111)
W20260809 07:51:43.855716 19873 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:43.856093 19873 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:43.856268 19873 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:43.864691 19873 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:43.864790 19873 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:43.864882 19873 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:43.864952 19873 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:43.876677 19873 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=33963
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:43.878456 19873 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:43.879854 19873 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:43.888419 19882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:43.889041 19883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:43.889503 19886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:43.889882 19873 server_base.cc:1061] running on GCE node
I20260809 07:51:43.890601 19873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:43.891935 19873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:43.893179 19873 hybrid_clock.cc:648] HybridClock initialized: now 1786261903893131 us; error 63 us; skew 500 ppm
I20260809 07:51:43.893576 19873 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:43 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261903, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:43.897104 19873 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:43.898806 19873 webserver.cc:533] Webserver started at http://127.26.37.62:33963/ using document root <none> and password file <none>
I20260809 07:51:43.899478 19873 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:43.899572 19873 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:43.905001 19873 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:43.907908 19895 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:43.909207 19873 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:51:43.909408 19873 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "7ba91afbe1cc4dcf9372df3276a56035"
format_stamp: "Formatted at 2026-08-09 07:51:30 on dist-test-slave-bw88"
I20260809 07:51:43.910099 19873 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:43.930140 19873 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:43.934091 19873 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:43.934355 19873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:43.955739 19873 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:45345
I20260809 07:51:43.955778 19967 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:45345 every 8 connection(s)
I20260809 07:51:43.957479 19873 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:43.964880 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19873
I20260809 07:51:43.965173 19968 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:43.965266 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19230
I20260809 07:51:43.969306 19968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap starting.
Aug 09 07:51:43 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:51:43.984463 19430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:43.989574 19968 log.cc:826] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:43.998488 19968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:43.999053 19968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: Bootstrap complete.
I20260809 07:51:44.006328 19968 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:44.007054 19968 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba91afbe1cc4dcf9372df3276a56035, State: Initialized, Role: FOLLOWER
I20260809 07:51:44.007891 19968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:44.008790 19976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:44.009075 19976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:44.009830 19968 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:44.020473 19992 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:44.020674 19992 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: loading cluster ID for follower catalog manager: success
I20260809 07:51:44.027885 19992 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: acquiring CA information for follower catalog manager: success
I20260809 07:51:44.031291 19992 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:51:44 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261903, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:44.069059 19873 master_runner.cc:264] No leader master found from master 7ba91afbe1cc4dcf9372df3276a56035
I20260809 07:51:44.069216 19873 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:44.110116 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19230
W20260809 07:51:44.143081 19118 connection.cc:573] server connection from 127.26.37.1:39517 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:51:44.144575 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44493
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--webserver_port=41781
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:44.144614 19996 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:44.109108 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:59709 (local address 127.26.37.62:45345)
0809 07:51:44.129536 (+ 20428us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:44.129543 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:44.129569 (+ 26us) server_negotiation.cc:408] Connection header received
0809 07:51:44.129675 (+ 106us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:44.129682 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:44.129776 (+ 94us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:44.129894 (+ 118us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0809 07:51:44.143207 (+ 13313us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:59709: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":20267}
I20260809 07:51:44.347592 19860 heartbeater.cc:499] Master 127.26.37.62:45345 was elected leader, sending a full tablet report...
I20260809 07:51:44.360770 19923 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e1f3b1764de34c8ea92fc79d53991336" instance_seqno: 1786261903286717) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:38151; Asking this server to re-register.
I20260809 07:51:44.361846 19860 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:44.363675 19923 ts_manager.cc:194] Registered new tserver with Master: e1f3b1764de34c8ea92fc79d53991336 (127.26.37.3:35657)
I20260809 07:51:44.438077 20004 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7ba91afbe1cc4dcf9372df3276a56035)
I20260809 07:51:44.438359 20004 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:44.440728 20004 leader_election.cc:290] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7ba91afbe1cc4dcf9372df3276a56035 (127.26.37.62:45345), cd429eb1e5e24c54a0009ed345e3e291 (127.26.37.60:33215)
I20260809 07:51:44.462929 19151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd429eb1e5e24c54a0009ed345e3e291" is_pre_election: true
I20260809 07:51:44.462951 19936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7ba91afbe1cc4dcf9372df3276a56035" is_pre_election: true
I20260809 07:51:44.463625 19936 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c99e566e0f84663bf9e5e6f211a06bc in term 2.
I20260809 07:51:44.464288 18994 leader_election.cc:304] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ba91afbe1cc4dcf9372df3276a56035, 8c99e566e0f84663bf9e5e6f211a06bc; no voters: cd429eb1e5e24c54a0009ed345e3e291
I20260809 07:51:44.464752 20004 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:51:44.464885 20004 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7ba91afbe1cc4dcf9372df3276a56035)
I20260809 07:51:44.464990 20004 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:44.467257 20004 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:44.467952 20004 leader_election.cc:290] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [CANDIDATE]: Term 3 election: Requested vote from peers 7ba91afbe1cc4dcf9372df3276a56035 (127.26.37.62:45345), cd429eb1e5e24c54a0009ed345e3e291 (127.26.37.60:33215)
I20260809 07:51:44.468421 19936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7ba91afbe1cc4dcf9372df3276a56035"
I20260809 07:51:44.468658 19936 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:44.468896 19151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd429eb1e5e24c54a0009ed345e3e291"
I20260809 07:51:44.473151 19936 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c99e566e0f84663bf9e5e6f211a06bc in term 3.
I20260809 07:51:44.473686 18994 leader_election.cc:304] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ba91afbe1cc4dcf9372df3276a56035, 8c99e566e0f84663bf9e5e6f211a06bc; no voters: cd429eb1e5e24c54a0009ed345e3e291
I20260809 07:51:44.474023 20004 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:51:44.474603 20004 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [term 3 LEADER]: Becoming Leader. State: Replica: 8c99e566e0f84663bf9e5e6f211a06bc, State: Running, Role: LEADER
I20260809 07:51:44.475106 20004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:44.477000 20009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c99e566e0f84663bf9e5e6f211a06bc. Latest consensus state: current_term: 3 leader_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:44.477247 20009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:44.478251 20011 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:51:44.480330 20011 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:51:44.481165 20011 catalog_manager.cc:1295] Loaded cluster ID: b38e02c31f774b589ae0861c0eb14fa1
I20260809 07:51:44.481274 20011 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:51:44.482465 20011 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:51:44.483477 20011 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc: Loaded TSK: 0
I20260809 07:51:44.484382 20011 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260809 07:51:44.568500 20000 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:44.568800 20000 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:44.568960 20000 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:44.577260 20000 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:44.577529 20000 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:44.588078 20000 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41781
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:44.589875 20000 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:44.591253 20000 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:44.601600 20022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:44.601866 20027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:44.601600 20017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:44.603597 20000 server_base.cc:1061] running on GCE node
I20260809 07:51:44.604460 20000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:44.605772 20000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:44.607038 20000 hybrid_clock.cc:648] HybridClock initialized: now 1786261904606975 us; error 50 us; skew 500 ppm
I20260809 07:51:44.607412 20000 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:44 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261904, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:44.611398 20000 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:44.613221 20000 webserver.cc:533] Webserver started at http://127.26.37.1:41781/ using document root <none> and password file <none>
I20260809 07:51:44.614243 20000 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:44.614396 20000 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:44.620229 20000 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 07:51:44.623189 20040 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:44.624677 20000 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:51:44.624876 20000 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:44.625834 20000 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:44.657027 20000 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:44.660846 20000 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:44.661077 20000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:44.662377 20000 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:44.663949 20000 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:44.664049 20000 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:44.664158 20000 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:44.664213 20000 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:44.699784 20000 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44493
I20260809 07:51:44.699872 20205 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44493 every 8 connection(s)
I20260809 07:51:44.701383 20000 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:44.709494 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20000
Aug 09 07:51:44 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261904, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:44 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261904, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:44.731318 20207 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33215
I20260809 07:51:44.731747 20207 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:44.732862 20207 heartbeater.cc:507] Master 127.26.37.60:33215 requested a full tablet report, sending...
Aug 09 07:51:44 dist-test-slave-bw88 krb5kdc[18808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261904, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:44.735050 19139 ts_manager.cc:194] Re-registered known tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:44493)
I20260809 07:51:44.738948 20208 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:44.739122 20208 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:44.739701 20208 heartbeater.cc:507] Master 127.26.37.62:45345 requested a full tablet report, sending...
I20260809 07:51:44.741058 19923 ts_manager.cc:194] Registered new tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:44493)
I20260809 07:51:44.744441 20209 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37019
I20260809 07:51:44.744598 20209 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:44.745024 20209 heartbeater.cc:507] Master 127.26.37.61:37019 requested a full tablet report, sending...
I20260809 07:51:44.746165 19013 ts_manager.cc:194] Re-registered known tserver with Master: 50fd01581b4b41c4b61a9bbff33217b3 (127.26.37.1:44493)
I20260809 07:51:44.747825 19013 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:39591
I20260809 07:51:44.766640 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20000
W20260809 07:51:44.767932 20224 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:44.763702 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.26.37.1:44493)
0809 07:51:44.764598 (+ 896us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:44.764609 (+ 11us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:44.764649 (+ 40us) server_negotiation.cc:408] Connection header received
0809 07:51:44.764912 (+ 263us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:44.764924 (+ 12us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:44.765237 (+ 313us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:44.765494 (+ 257us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:44.766474 (+ 980us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49606: BlockingRecv error: recv got EOF from 127.0.0.1:49606 (error 108)
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":587,"threads_started":1}
W20260809 07:51:44.769102 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20000: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:44493: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:44.769260 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20000
I20260809 07:51:44.789070 19646 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45345
I20260809 07:51:44.789402 19646 heartbeater.cc:499] Master 127.26.37.62:45345 was elected leader, sending a full tablet report...
I20260809 07:51:44.790506 19923 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "25e0f3d3c31d4d218833ac36648d4a07" instance_seqno: 1786261902685480) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:36231; Asking this server to re-register.
I20260809 07:51:44.791167 19646 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:44.792307 19923 ts_manager.cc:194] Registered new tserver with Master: 25e0f3d3c31d4d218833ac36648d4a07 (127.26.37.2:37715)
I20260809 07:51:44.807157 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44493
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--webserver_port=41781
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:44.808161 20228 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7ba91afbe1cc4dcf9372df3276a56035)
I20260809 07:51:44.808416 20228 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } }
I20260809 07:51:44.810523 20228 leader_election.cc:290] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7ba91afbe1cc4dcf9372df3276a56035 (127.26.37.62:45345), 8c99e566e0f84663bf9e5e6f211a06bc (127.26.37.61:37019)
I20260809 07:51:44.811360 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd429eb1e5e24c54a0009ed345e3e291" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" is_pre_election: true
I20260809 07:51:44.818037 19936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd429eb1e5e24c54a0009ed345e3e291" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7ba91afbe1cc4dcf9372df3276a56035" is_pre_election: true
I20260809 07:51:44.818401 19936 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd429eb1e5e24c54a0009ed345e3e291 in current term 3: Already voted for candidate 8c99e566e0f84663bf9e5e6f211a06bc in this term.
I20260809 07:51:44.819106 19118 leader_election.cc:304] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd429eb1e5e24c54a0009ed345e3e291; no voters: 7ba91afbe1cc4dcf9372df3276a56035, 8c99e566e0f84663bf9e5e6f211a06bc
I20260809 07:51:44.819535 20228 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:44.821786 20228 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:51:45.015375 19936 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [term 3 FOLLOWER]: Refusing update from remote peer 8c99e566e0f84663bf9e5e6f211a06bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:51:45.016353 20232 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:51:45.020474 20233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c99e566e0f84663bf9e5e6f211a06bc. Latest consensus state: current_term: 3 leader_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:45.020843 20233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:45.022776 20233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:45.023008 20233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba91afbe1cc4dcf9372df3276a56035 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:45.022980 19151 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [term 3 FOLLOWER]: Refusing update from remote peer 8c99e566e0f84663bf9e5e6f211a06bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:51:45.023970 20241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:45.024329 20241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:45.029798 20236 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:51:45.034739 20228 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c99e566e0f84663bf9e5e6f211a06bc. Latest consensus state: current_term: 3 leader_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:45.034651 20250 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:45.035012 20228 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:45.035036 20250 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd429eb1e5e24c54a0009ed345e3e291 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:51:45.170387 20231 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:45.170706 20231 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:45.170895 20231 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:51:45.170981 20231 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:45.179364 20231 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:45.179636 20231 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:45.190157 20231 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41781
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:45.191893 20231 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:45.193281 20231 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:45.203958 20257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:45.203926 20256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:45.204456 20260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:45.205165 20231 server_base.cc:1061] running on GCE node
I20260809 07:51:45.205847 20231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:45.206765 20231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:45.207973 20231 hybrid_clock.cc:648] HybridClock initialized: now 1786261905207900 us; error 59 us; skew 500 ppm
I20260809 07:51:45.208225 20231 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:45 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261905, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:45.211968 20231 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:45.213747 20231 webserver.cc:533] Webserver started at http://127.26.37.1:41781/ using document root <none> and password file <none>
I20260809 07:51:45.214535 20231 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:45.214679 20231 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:45.220405 20231 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:51:45.223249 20273 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:45.224642 20231 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:51:45.224817 20231 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:45.225734 20231 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:45.249526 20231 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:45.256783 20231 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:45.257093 20231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:45.258067 20231 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:45.259615 20231 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:45.259719 20231 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:45.259860 20231 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:45.259930 20231 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:51:45.260318 20231 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:51:45.260409 20231 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:51:45.260497 20231 fault_injection.cc:59] injecting fault for kudu (pid 20231): FLAGS_fault_before_start (process will exit)
I20260809 07:51:45.287737 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44493
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--webserver_port=41781
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:45.637276 20283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba91afbe1cc4dcf9372df3276a56035" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45345 } } peers { permanent_uuid: "8c99e566e0f84663bf9e5e6f211a06bc" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37019 } } peers { permanent_uuid: "cd429eb1e5e24c54a0009ed345e3e291" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33215 } } }
I20260809 07:51:45.637566 20283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c99e566e0f84663bf9e5e6f211a06bc [sys.catalog]: This master's current role is: LEADER
W20260809 07:51:45.642135 20282 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:45.642454 20282 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:45.642561 20282 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:45.650840 20282 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:45.651019 20282 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:45.661412 20282 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41781
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45345,127.26.37.61:37019,127.26.37.60:33215
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:45.663008 20282 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:45.664376 20282 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:45.674413 20294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:45.674439 20293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:45.674856 20297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:45.675801 20282 server_base.cc:1061] running on GCE node
I20260809 07:51:45.676472 20282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:45.677469 20282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:45.678705 20282 hybrid_clock.cc:648] HybridClock initialized: now 1786261905678648 us; error 47 us; skew 500 ppm
I20260809 07:51:45.679010 20282 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:45 dist-test-slave-bw88 krb5kdc[18808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261905, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:45.682646 20282 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:45.684156 20282 webserver.cc:533] Webserver started at http://127.26.37.1:41781/ using document root <none> and password file <none>
I20260809 07:51:45.684957 20282 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:45.685057 20282 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:45.690663 20282 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:51:45.693760 20308 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:45.695605 20282 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:51:45.695787 20282 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "50fd01581b4b41c4b61a9bbff33217b3"
format_stamp: "Formatted at 2026-08-09 07:51:33 on dist-test-slave-bw88"
I20260809 07:51:45.696745 20282 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:45.709616 20282 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:45.713032 20282 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:45.713325 20282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:45.714444 20282 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:45.716042 20282 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:45.716156 20282 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:45.716290 20282 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:45.716387 20282 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:45.750902 20282 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44493
I20260809 07:51:45.751003 20460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44493 every 8 connection(s)
I20260809 07:51:45.752522 20282 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:45.756093 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20282
I20260809 07:51:45.756434 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20282
I20260809 07:51:45.762864 19652 heartbeater.cc:499] Master 127.26.37.61:37019 was elected leader, sending a full tablet report...
W20260809 07:51:45.781734 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20282: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:44493: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:45.781904 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20282
I20260809 07:51:45.806586 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19458
W20260809 07:51:45.806782 20467 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:45.757007 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:56557 (local address 127.26.37.62:45345)
0809 07:51:45.778577 (+ 21570us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:45.778583 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:45.779132 (+ 549us) server_negotiation.cc:408] Connection header received
0809 07:51:45.780364 (+ 1232us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:45.780371 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:45.786064 (+ 5693us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:45.786157 (+ 93us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:45.806288 (+ 20131us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:56557: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":21440,"thread_start_us":21347,"threads_started":1}
W20260809 07:51:45.807334 20465 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:45.756018 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:35709 (local address 127.26.37.60:33215)
0809 07:51:45.756568 (+ 550us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:45.756573 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:45.775548 (+ 18975us) server_negotiation.cc:408] Connection header received
0809 07:51:45.779605 (+ 4057us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:45.779614 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:45.779721 (+ 107us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:45.779853 (+ 132us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:45.785636 (+ 5783us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:45.787301 (+ 1665us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:45.806353 (+ 19052us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:35709: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":682,"threads_started":1}
W20260809 07:51:45.807600 20475 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:45.779336 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:55667 (local address 127.26.37.61:37019)
0809 07:51:45.780078 (+ 742us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:45.780082 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:45.806225 (+ 26143us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:55667: BlockingRecv error: recv got EOF from 127.26.37.1:55667 (error 108)
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":657,"threads_started":1}
W20260809 07:51:45.809429 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19458: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:37715: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:45.809576 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19458
I20260809 07:51:45.834335 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19671
W20260809 07:51:45.837013 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19671: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:35657: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:45.837181 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19671
I20260809 07:51:45.859973 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19873
W20260809 07:51:45.862411 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19873: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:45345: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:45.862535 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19873
W20260809 07:51:45.862543 20467 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:45.860836 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37890 (local address 127.26.37.62:45345)
0809 07:51:45.861016 (+ 180us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:45.861023 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:45.861102 (+ 79us) server_negotiation.cc:408] Connection header received
0809 07:51:45.861402 (+ 300us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:45.861408 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:45.861476 (+ 68us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:45.861602 (+ 126us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:45.862337 (+ 735us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37890: BlockingRecv error: recv got EOF from 127.0.0.1:37890 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 07:51:45.890776 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18967
I20260809 07:51:45.891990 19058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:46.002815 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18967
I20260809 07:51:46.032076 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19092
I20260809 07:51:46.033466 19175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:46.142498 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19092
2026-08-09T07:51:46Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/7 (18342 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/8
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:51:46 dist-test-slave-bw88 krb5kdc[20487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:46 dist-test-slave-bw88 krb5kdc[20487](info): set up 2 sockets
Aug 09 07:51:46 dist-test-slave-bw88 krb5kdc[20487](info): commencing operation
krb5kdc: starting...
W20260809 07:51:48.365868 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.171s user 0.005s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:51:48 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:51:48Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:48Z Disabled control of system clock
2026-08-09T07:51:48Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:48Z Disabled control of system clock
2026-08-09T07:51:48Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:48Z Disabled control of system clock
2026-08-09T07:51:48Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:48Z Disabled control of system clock
2026-08-09T07:51:48Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:48.812216 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41249
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:49.155887 20520 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:49.156193 20520 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:49.156278 20520 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:49.164227 20520 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:49.164324 20520 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:49.164376 20520 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:49.164414 20520 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:49.174875 20520 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:49.176460 20520 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:49.177806 20520 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:49.186914 20526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:49.187126 20527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:49.187599 20529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:49.188761 20520 server_base.cc:1061] running on GCE node
I20260809 07:51:49.189656 20520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:49.191359 20520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:49.192582 20520 hybrid_clock.cc:648] HybridClock initialized: now 1786261909192459 us; error 93 us; skew 500 ppm
I20260809 07:51:49.192963 20520 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:49 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261909, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:49.196152 20520 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:49.197857 20520 webserver.cc:533] Webserver started at http://127.26.37.62:38687/ using document root <none> and password file <none>
I20260809 07:51:49.198617 20520 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:49.198765 20520 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:49.199102 20520 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:49.201576 20520 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "cd2b7bfe6a214907a7ee705b58948d46"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:49.202386 20520 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "cd2b7bfe6a214907a7ee705b58948d46"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:49.207791 20520 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260809 07:51:49.211359 20538 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:49.212844 20520 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:49.213061 20520 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cd2b7bfe6a214907a7ee705b58948d46"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:49.213303 20520 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:49.231431 20520 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:49.235196 20520 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:49.235515 20520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:49.257033 20520 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41249
I20260809 07:51:49.257052 20614 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41249 every 8 connection(s)
I20260809 07:51:49.258694 20520 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:49.263135 20616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:49.268878 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20520
I20260809 07:51:49.269112 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:51:49.269418 20616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:49 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261909, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:49.294894 20616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:49.296293 20539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:40663: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
W20260809 07:51:49.298024 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:40663: Network error: Client connection negotiation failed: client connection to 127.26.37.61:40663: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:49.347410 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } attempt: 1
W20260809 07:51:49.349535 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:40663: Network error: Client connection negotiation failed: client connection to 127.26.37.61:40663: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:49.384403 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:40663
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:49.417801 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } attempt: 2
W20260809 07:51:49.419824 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:40663: Network error: Client connection negotiation failed: client connection to 127.26.37.61:40663: connect: Connection refused (error 111)
I20260809 07:51:49.511150 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } attempt: 3
W20260809 07:51:49.513970 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:40663: Network error: Client connection negotiation failed: client connection to 127.26.37.61:40663: connect: Connection refused (error 111)
I20260809 07:51:49.657294 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } attempt: 4
W20260809 07:51:49.659478 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:40663: Network error: Client connection negotiation failed: client connection to 127.26.37.61:40663: connect: Connection refused (error 111)
W20260809 07:51:49.734620 20627 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:49.734922 20627 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:49.735036 20627 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:49.743095 20627 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:49.743181 20627 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:49.743235 20627 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:49.743283 20627 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:49.753754 20627 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:49.755267 20627 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:49.756577 20627 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:49.764508 20632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:49.764557 20635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:49.765311 20637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:49.765727 20627 server_base.cc:1061] running on GCE node
I20260809 07:51:49.766575 20627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:49.768215 20627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:49.769439 20627 hybrid_clock.cc:648] HybridClock initialized: now 1786261909769303 us; error 93 us; skew 500 ppm
I20260809 07:51:49.769793 20627 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:49 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261909, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:49.772799 20627 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:51:49.774426 20627 webserver.cc:533] Webserver started at http://127.26.37.61:33273/ using document root <none> and password file <none>
I20260809 07:51:49.775182 20627 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:49.775321 20627 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:49.775671 20627 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:49.778221 20627 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "f8cd98f2b2c8473fbdcf497a37e26656"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:49.779035 20627 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "f8cd98f2b2c8473fbdcf497a37e26656"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:49.784651 20627 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:51:49.787923 20647 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:49.789245 20627 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:49.789455 20627 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "f8cd98f2b2c8473fbdcf497a37e26656"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:49.789672 20627 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:49.800868 20627 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:49.804559 20627 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:49.804852 20627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:49.826241 20627 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:40663
I20260809 07:51:49.826283 20723 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:40663 every 8 connection(s)
I20260809 07:51:49.827733 20627 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:51:49.832087 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:49.832197 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20627
I20260809 07:51:49.832429 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:51:49.841187 20724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:49 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261909, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
I20260809 07:51:49.864723 20724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
Aug 09 07:51:49 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261909, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:49.882568 20724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:49.883656 20650 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
W20260809 07:51:49.885258 20724 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:51:49.934688 20724 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 1
W20260809 07:51:49.936864 20724 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:49.956619 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:42841
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:49.958731 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } attempt: 5
Aug 09 07:51:49 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261909, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:49.973060 20616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:51:49.975078 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
I20260809 07:51:50.005062 20724 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 2
W20260809 07:51:50.007162 20724 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
I20260809 07:51:50.026335 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 1
W20260809 07:51:50.028239 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
I20260809 07:51:50.096503 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 2
I20260809 07:51:50.098408 20724 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 3
W20260809 07:51:50.099316 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
W20260809 07:51:50.100442 20724 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
I20260809 07:51:50.205561 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 3
W20260809 07:51:50.207858 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
I20260809 07:51:50.243680 20724 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 4
W20260809 07:51:50.245898 20724 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
W20260809 07:51:50.301314 20738 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:50.301656 20738 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:50.301846 20738 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:50.309953 20738 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:50.310048 20738 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:50.310144 20738 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:50.310221 20738 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:50.320657 20738 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:42841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:50.322304 20738 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:50.323669 20738 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:50.331717 20744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:50.331773 20745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:50.332640 20749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:50.332918 20738 server_base.cc:1061] running on GCE node
I20260809 07:51:50.333729 20738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:50.335181 20738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:50.336385 20738 hybrid_clock.cc:648] HybridClock initialized: now 1786261910336231 us; error 124 us; skew 500 ppm
I20260809 07:51:50.336750 20738 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:50.339852 20738 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:51:50.341598 20738 webserver.cc:533] Webserver started at http://127.26.37.60:36891/ using document root <none> and password file <none>
I20260809 07:51:50.342350 20738 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:50.342525 20738 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:50.342888 20738 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:50.345440 20738 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "4e853cca2d0243d7ba8334d9199b4815"
format_stamp: "Formatted at 2026-08-09 07:51:50 on dist-test-slave-bw88"
I20260809 07:51:50.346328 20738 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "4e853cca2d0243d7ba8334d9199b4815"
format_stamp: "Formatted at 2026-08-09 07:51:50 on dist-test-slave-bw88"
I20260809 07:51:50.351749 20738 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:51:50.355201 20761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:50.356462 20738 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:50.356700 20738 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "4e853cca2d0243d7ba8334d9199b4815"
format_stamp: "Formatted at 2026-08-09 07:51:50 on dist-test-slave-bw88"
I20260809 07:51:50.356930 20738 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:50.385102 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 4
W20260809 07:51:50.387184 20616 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111)
I20260809 07:51:50.393947 20738 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:50.400970 20738 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:50.401291 20738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:50.421976 20738 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:42841
I20260809 07:51:50.422058 20832 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:42841 every 8 connection(s)
I20260809 07:51:50.423527 20738 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:51:50.424170 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20738
I20260809 07:51:50.424379 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:51:50.429201 20833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:50.437072 20833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:50.458084 20833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:50.471985 20833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261910, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:50.488698 20833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Bootstrap starting.
I20260809 07:51:50.491865 20833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:50.492899 20833 log.cc:826] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:50.495389 20833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: No bootstrap required, opened a new log
I20260809 07:51:50.500679 20833 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.500937 20833 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:50.501049 20833 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e853cca2d0243d7ba8334d9199b4815, State: Initialized, Role: FOLLOWER
I20260809 07:51:50.501701 20833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.502553 20848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.502826 20848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:50.503531 20833 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:51:50.514451 20868 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:50.514593 20868 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:50.511785 20738 master_runner.cc:187] Error getting master registration for 127.26.37.62:41249: OK, instance_id { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" instance_seqno: 1786261909251202 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:51:50.517144 20738 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:50.545166 20724 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 5
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261909, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:50.562836 20724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: Bootstrap starting.
I20260809 07:51:50.566948 20724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:50.568035 20724 log.cc:826] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:50.570538 20724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: No bootstrap required, opened a new log
I20260809 07:51:50.575683 20724 raft_consensus.cc:359] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.575953 20724 raft_consensus.cc:385] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:50.576097 20724 raft_consensus.cc:740] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cd98f2b2c8473fbdcf497a37e26656, State: Initialized, Role: FOLLOWER
I20260809 07:51:50.576776 20724 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.577802 20871 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.578078 20871 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:50.579630 20724 sys_catalog.cc:565] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:51:50.590372 20887 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:50.590591 20887 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:50.587482 20627 master_runner.cc:187] Error getting master registration for 127.26.37.62:41249: OK, instance_id { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" instance_seqno: 1786261909251202 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:51:50.594437 20627 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:51:50.653549 20772 tablet.cc:2406] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:50.664403 20616 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } attempt: 5
Aug 09 07:51:50 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261909, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:50.687085 20616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap starting.
I20260809 07:51:50.691232 20616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:50.692659 20616 log.cc:826] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:50.696187 20616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: No bootstrap required, opened a new log
I20260809 07:51:50.698459 20871 raft_consensus.cc:493] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:50.698743 20871 raft_consensus.cc:515] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.700472 20871 leader_election.cc:290] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd2b7bfe6a214907a7ee705b58948d46 (127.26.37.62:41249), 4e853cca2d0243d7ba8334d9199b4815 (127.26.37.60:42841)
I20260809 07:51:50.701831 20798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e853cca2d0243d7ba8334d9199b4815" is_pre_election: true
I20260809 07:51:50.702440 20798 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8cd98f2b2c8473fbdcf497a37e26656 in term 0.
I20260809 07:51:50.703119 20650 leader_election.cc:304] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e853cca2d0243d7ba8334d9199b4815, f8cd98f2b2c8473fbdcf497a37e26656; no voters:
I20260809 07:51:50.703581 20871 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:51:50.703720 20871 raft_consensus.cc:493] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:50.703824 20871 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:50.707319 20871 raft_consensus.cc:515] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.709599 20798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e853cca2d0243d7ba8334d9199b4815"
I20260809 07:51:50.709805 20798 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:50.710239 20616 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.710857 20616 raft_consensus.cc:385] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:50.711138 20616 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2b7bfe6a214907a7ee705b58948d46, State: Initialized, Role: FOLLOWER
I20260809 07:51:50.712507 20616 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.713405 20798 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8cd98f2b2c8473fbdcf497a37e26656 in term 1.
I20260809 07:51:50.714058 20650 leader_election.cc:304] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e853cca2d0243d7ba8334d9199b4815, f8cd98f2b2c8473fbdcf497a37e26656; no voters:
I20260809 07:51:50.716465 20871 leader_election.cc:290] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [CANDIDATE]: Term 1 election: Requested vote from peers cd2b7bfe6a214907a7ee705b58948d46 (127.26.37.62:41249), 4e853cca2d0243d7ba8334d9199b4815 (127.26.37.60:42841)
I20260809 07:51:50.717379 20871 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:50.717986 20871 raft_consensus.cc:697] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 1 LEADER]: Becoming Leader. State: Replica: f8cd98f2b2c8473fbdcf497a37e26656, State: Running, Role: LEADER
I20260809 07:51:50.722965 20871 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:50.743403 20893 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8cd98f2b2c8473fbdcf497a37e26656. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.744014 20893 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:50.747926 20892 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.749264 20892 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:50.749229 20578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd2b7bfe6a214907a7ee705b58948d46" is_pre_election: true
I20260809 07:51:50.749789 20578 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8cd98f2b2c8473fbdcf497a37e26656 in term 0.
I20260809 07:51:50.750381 20577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd2b7bfe6a214907a7ee705b58948d46"
I20260809 07:51:50.750626 20577 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:50.751012 20895 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:51:50.753284 20895 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:51:50.753969 20577 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8cd98f2b2c8473fbdcf497a37e26656 in term 1.
I20260809 07:51:50.754793 20616 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:50.772616 20798 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 1 FOLLOWER]: Refusing update from remote peer f8cd98f2b2c8473fbdcf497a37e26656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:50.773656 20893 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:50.780946 20577 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Refusing update from remote peer f8cd98f2b2c8473fbdcf497a37e26656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:50.782354 20893 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:50.802454 20848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8cd98f2b2c8473fbdcf497a37e26656. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.802760 20848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:50.805883 20898 mvcc.cc:204] Tried to move back new op lower bound from 7316528786512658432 to 7316528786342600704. Current Snapshot: MvccSnapshot[applied={T|T < 7316528786512658432}]
I20260809 07:51:50.806815 20892 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8cd98f2b2c8473fbdcf497a37e26656. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.807108 20892 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:50.811859 20900 mvcc.cc:204] Tried to move back new op lower bound from 7316528786512658432 to 7316528786342600704. Current Snapshot: MvccSnapshot[applied={T|T < 7316528786512658432}]
I20260809 07:51:50.813810 20893 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.814426 20893 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:50.814667 20893 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.814862 20893 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:50.816468 20895 catalog_manager.cc:1383] Generated new cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:51:50.816574 20895 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:51:50.817862 20848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.818100 20848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:50.862146 20892 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:50.862439 20892 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:50.883236 20895 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:51:50.887488 20895 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:51:50.893117 20923 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:51:50.893281 20923 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: loading cluster ID for follower catalog manager: success
I20260809 07:51:50.897372 20923 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: acquiring CA information for follower catalog manager: success
W20260809 07:51:50.898777 20923 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260809 07:51:50.923676 20895 catalog_manager.cc:6092] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: Generated new TSK 0
I20260809 07:51:50.925016 20895 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:51.106901 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:51.479215 20929 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:51.479504 20929 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:51.479604 20929 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:51.487921 20929 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:51.488090 20929 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:51.498387 20929 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:51.499908 20929 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:51.501245 20929 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:51.511848 20938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:51.511801 20936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:51.512465 20935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:51.512720 20929 server_base.cc:1061] running on GCE node
I20260809 07:51:51.513396 20929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:51.514444 20929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:51.515663 20929 hybrid_clock.cc:648] HybridClock initialized: now 1786261911515495 us; error 137 us; skew 500 ppm
I20260809 07:51:51.516014 20929 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:51:51.516633 20868 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:51:51.516739 20868 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: loading cluster ID for follower catalog manager: success
Aug 09 07:51:51 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261911, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:51.519546 20929 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:51.519558 20868 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: acquiring CA information for follower catalog manager: success
I20260809 07:51:51.521519 20868 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:51.522634 20929 webserver.cc:533] Webserver started at http://127.26.37.1:33771/ using document root <none> and password file <none>
I20260809 07:51:51.523353 20929 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:51.523504 20929 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:51.523837 20929 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:51.526288 20929 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:51:51.527091 20929 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:51:51.532744 20929 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:51:51.536170 20949 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:51.537523 20929 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:51:51.537711 20929 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:51:51.537925 20929 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:51.568548 20929 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:51.572770 20929 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:51.573097 20929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:51.574194 20929 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:51.575896 20929 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:51.576013 20929 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:51.576167 20929 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:51.576241 20929 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:51.612040 20929 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34779
I20260809 07:51:51.612095 21114 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34779 every 8 connection(s)
I20260809 07:51:51.613574 20929 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:51.617156 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20929
I20260809 07:51:51.617420 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:51:51 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261911, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:51 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261911, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:51 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261911, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:51:51.668114 21122 heartbeater.cc:344] Connected to a master server at 127.26.37.61:40663
I20260809 07:51:51.668134 21120 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
I20260809 07:51:51.668654 21115 heartbeater.cc:344] Connected to a master server at 127.26.37.60:42841
I20260809 07:51:51.668655 21122 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:51.668653 21120 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:51.668833 21115 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:51.670140 21120 heartbeater.cc:507] Master 127.26.37.62:41249 requested a full tablet report, sending...
I20260809 07:51:51.670113 21122 heartbeater.cc:507] Master 127.26.37.61:40663 requested a full tablet report, sending...
I20260809 07:51:51.670337 21115 heartbeater.cc:507] Master 127.26.37.60:42841 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.2@KRBTEST.COM" created.
I20260809 07:51:51.672822 20668 ts_manager.cc:194] Registered new tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:34779)
I20260809 07:51:51.672868 20566 ts_manager.cc:194] Registered new tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:34779)
I20260809 07:51:51.674631 20785 ts_manager.cc:194] Registered new tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:34779)
I20260809 07:51:51.676186 20668 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:36949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:51.760994 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:51.901576 20923 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:51:52.114704 21144 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:52.115090 21144 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:52.115252 21144 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:52.124408 21144 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:52.124650 21144 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:51:52.135499 21144 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:52.137197 21144 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:52.138604 21144 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:52.148010 21150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:52.148006 21151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:52.148756 21154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:52.149212 21144 server_base.cc:1061] running on GCE node
I20260809 07:51:52.149856 21144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:52.151259 21144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:52.152479 21144 hybrid_clock.cc:648] HybridClock initialized: now 1786261912152292 us; error 148 us; skew 500 ppm
I20260809 07:51:52.152812 21144 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:52 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:52.159067 21144 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:51:52.160665 21144 webserver.cc:533] Webserver started at http://127.26.37.2:38311/ using document root <none> and password file <none>
I20260809 07:51:52.161523 21144 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:52.161659 21144 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:52.162034 21144 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:52.164564 21144 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "e12904b027a144d3a9245687086d3921"
format_stamp: "Formatted at 2026-08-09 07:51:52 on dist-test-slave-bw88"
I20260809 07:51:52.165457 21144 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "e12904b027a144d3a9245687086d3921"
format_stamp: "Formatted at 2026-08-09 07:51:52 on dist-test-slave-bw88"
I20260809 07:51:52.170879 21144 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:51:52.174280 21166 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:52.175560 21144 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:51:52.175745 21144 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "e12904b027a144d3a9245687086d3921"
format_stamp: "Formatted at 2026-08-09 07:51:52 on dist-test-slave-bw88"
I20260809 07:51:52.175958 21144 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:52.215749 21144 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:52.220544 21144 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:52.220852 21144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:52.221979 21144 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:52.223686 21144 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:52.223793 21144 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:52.223935 21144 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:52.224009 21144 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:52.258616 21144 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:44113
I20260809 07:51:52.258728 21329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:44113 every 8 connection(s)
I20260809 07:51:52.260128 21144 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:52.269965 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21144
I20260809 07:51:52.270227 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:51:52 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:52 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:51:52 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:52.307204 21334 heartbeater.cc:344] Connected to a master server at 127.26.37.61:40663
I20260809 07:51:52.307690 21334 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:52.308609 21332 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:51:52.308761 21332 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:52.308849 21334 heartbeater.cc:507] Master 127.26.37.61:40663 requested a full tablet report, sending...
I20260809 07:51:52.309362 21332 heartbeater.cc:507] Master 127.26.37.62:41249 requested a full tablet report, sending...
I20260809 07:51:52.310670 20668 ts_manager.cc:194] Registered new tserver with Master: e12904b027a144d3a9245687086d3921 (127.26.37.2:44113)
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
I20260809 07:51:52.310803 20566 ts_manager.cc:194] Registered new tserver with Master: e12904b027a144d3a9245687086d3921 (127.26.37.2:44113)
I20260809 07:51:52.311846 20668 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:40399
I20260809 07:51:52.313472 21331 heartbeater.cc:344] Connected to a master server at 127.26.37.60:42841
I20260809 07:51:52.313607 21331 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:52.313889 21331 heartbeater.cc:507] Master 127.26.37.60:42841 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.3@KRBTEST.COM" created.
I20260809 07:51:52.315150 20785 ts_manager.cc:194] Registered new tserver with Master: e12904b027a144d3a9245687086d3921 (127.26.37.2:44113)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:52.405390 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:52.679452 21122 heartbeater.cc:499] Master 127.26.37.61:40663 was elected leader, sending a full tablet report...
W20260809 07:51:52.803045 21347 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:52.803369 21347 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:52.803468 21347 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:52.812390 21347 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:52.812585 21347 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:51:52.823319 21347 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:52.824983 21347 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:52.826457 21347 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:52.836604 21354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:52.836606 21355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:52.837232 21358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:52.837922 21347 server_base.cc:1061] running on GCE node
I20260809 07:51:52.838557 21347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:52.839738 21347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:52.840989 21347 hybrid_clock.cc:648] HybridClock initialized: now 1786261912840813 us; error 140 us; skew 500 ppm
I20260809 07:51:52.841419 21347 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:52 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:52.845196 21347 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:51:52.847126 21347 webserver.cc:533] Webserver started at http://127.26.37.3:45215/ using document root <none> and password file <none>
I20260809 07:51:52.847900 21347 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:52.848050 21347 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:52.848412 21347 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:52.850950 21347 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "8cb20c1bb15e433c9cd550b59ce25b03"
format_stamp: "Formatted at 2026-08-09 07:51:52 on dist-test-slave-bw88"
I20260809 07:51:52.851730 21347 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "8cb20c1bb15e433c9cd550b59ce25b03"
format_stamp: "Formatted at 2026-08-09 07:51:52 on dist-test-slave-bw88"
I20260809 07:51:52.857437 21347 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:51:52.860929 21370 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:52.862457 21347 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:52.862660 21347 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "8cb20c1bb15e433c9cd550b59ce25b03"
format_stamp: "Formatted at 2026-08-09 07:51:52 on dist-test-slave-bw88"
I20260809 07:51:52.862838 21347 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:52.892498 21347 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:52.896909 21347 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:52.897255 21347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:52.898396 21347 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:52.900125 21347 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:52.900207 21347 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:52.900389 21347 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:52.900442 21347 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:52.938158 21347 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:43219
I20260809 07:51:52.938246 21532 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:43219 every 8 connection(s)
I20260809 07:51:52.939564 21347 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:52.947379 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21347
I20260809 07:51:52.947624 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:51:52 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:51:52 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:52 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261912, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:52.967110 21538 heartbeater.cc:344] Connected to a master server at 127.26.37.61:40663
I20260809 07:51:52.967540 21538 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:52.967834 21537 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:51:52.967990 21537 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:52.968371 21535 heartbeater.cc:344] Connected to a master server at 127.26.37.60:42841
I20260809 07:51:52.968534 21535 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:52.968855 21537 heartbeater.cc:507] Master 127.26.37.62:41249 requested a full tablet report, sending...
I20260809 07:51:52.969022 21535 heartbeater.cc:507] Master 127.26.37.60:42841 requested a full tablet report, sending...
I20260809 07:51:52.968827 21538 heartbeater.cc:507] Master 127.26.37.61:40663 requested a full tablet report, sending...
I20260809 07:51:52.971002 20785 ts_manager.cc:194] Registered new tserver with Master: 8cb20c1bb15e433c9cd550b59ce25b03 (127.26.37.3:43219)
I20260809 07:51:52.971002 20667 ts_manager.cc:194] Registered new tserver with Master: 8cb20c1bb15e433c9cd550b59ce25b03 (127.26.37.3:43219)
I20260809 07:51:52.971722 20566 ts_manager.cc:194] Registered new tserver with Master: 8cb20c1bb15e433c9cd550b59ce25b03 (127.26.37.3:43219)
I20260809 07:51:52.972223 20667 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:41617
I20260809 07:51:52.974169 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:52.981279 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:51:52.981523 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20520
I20260809 07:51:52.982373 20609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:53.094707 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20520
I20260809 07:51:53.125244 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41249
--webserver_interface=127.26.37.62
--webserver_port=38687
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:53.213703 20650 consensus_peers.cc:597] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 -> Peer cd2b7bfe6a214907a7ee705b58948d46 (127.26.37.62:41249): Couldn't send request to peer cd2b7bfe6a214907a7ee705b58948d46. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:41249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:51:53.314137 21332 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:41249 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:41249: connect: Connection refused (error 111)
I20260809 07:51:53.314615 21334 heartbeater.cc:499] Master 127.26.37.61:40663 was elected leader, sending a full tablet report...
W20260809 07:51:53.477680 21549 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:53.478000 21549 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:53.478099 21549 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:53.486622 21549 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:53.486717 21549 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:53.486774 21549 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:53.486826 21549 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:53.497413 21549 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=38687
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:53.499009 21549 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:53.500326 21549 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:53.508993 21562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:53.508963 21563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:53.509698 21565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:53.510635 21549 server_base.cc:1061] running on GCE node
I20260809 07:51:53.511379 21549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:53.513059 21549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:53.514300 21549 hybrid_clock.cc:648] HybridClock initialized: now 1786261913514144 us; error 125 us; skew 500 ppm
I20260809 07:51:53.514658 21549 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:53 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261913, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:53.518100 21549 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:53.519680 21549 webserver.cc:533] Webserver started at http://127.26.37.62:38687/ using document root <none> and password file <none>
I20260809 07:51:53.520411 21549 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:53.520567 21549 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:53.525913 21549 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:51:53.528721 21574 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:53.530118 21549 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:53.530303 21549 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cd2b7bfe6a214907a7ee705b58948d46"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:53.531020 21549 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:53.559350 21549 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:53.563802 21549 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:53.564102 21549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:53.585960 21549 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41249
I20260809 07:51:53.585980 21645 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41249 every 8 connection(s)
I20260809 07:51:53.587777 21549 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:53.592370 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21549
I20260809 07:51:53.592718 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20929
I20260809 07:51:53.596007 21646 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:53.600459 21646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap starting.
Aug 09 07:51:53 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:51:53.612782 21109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:53.621470 21646 log.cc:826] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:53.627079 21145 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:53.627367 21145 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:53.632287 21646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:53.633036 21646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap complete.
I20260809 07:51:53.648348 21646 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:53.649240 21646 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2b7bfe6a214907a7ee705b58948d46, State: Initialized, Role: FOLLOWER
I20260809 07:51:53.650081 21646 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:53.651018 21666 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:53.651415 21666 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:53.652320 21646 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:53.667232 21666 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8cd98f2b2c8473fbdcf497a37e26656. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:53.667553 21666 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:53.672868 21681 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:51:53.672983 21681 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: loading cluster ID for follower catalog manager: success
I20260809 07:51:53.677029 21681 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: acquiring CA information for follower catalog manager: success
I20260809 07:51:53.679507 21681 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:51:53 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261913, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:53.757117 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20929
I20260809 07:51:53.782156 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34779
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--webserver_port=33771
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:53.782533 21665 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:53.752198 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:47123 (local address 127.26.37.62:41249)
0809 07:51:53.752391 (+ 193us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:53.752397 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:53.756980 (+ 4583us) server_negotiation.cc:408] Connection header received
0809 07:51:53.758227 (+ 1247us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:53.758235 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:53.758334 (+ 99us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:53.758409 (+ 75us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0809 07:51:53.781113 (+ 22704us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:47123: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 07:51:53.974702 21538 heartbeater.cc:499] Master 127.26.37.61:40663 was elected leader, sending a full tablet report...
I20260809 07:51:53.984756 21594 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8cb20c1bb15e433c9cd550b59ce25b03" instance_seqno: 1786261912929190) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:38395; Asking this server to re-register.
I20260809 07:51:53.985647 21537 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:53.987329 21594 ts_manager.cc:194] Registered new tserver with Master: 8cb20c1bb15e433c9cd550b59ce25b03 (127.26.37.3:43219)
I20260809 07:51:54.039180 21658 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:54.039512 21658 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: LEADER
W20260809 07:51:54.171811 21688 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:54.172155 21688 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:54.172276 21688 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:54.181316 21688 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:54.181540 21688 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:54.191994 21688 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33771
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:54.193704 21688 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:54.195253 21688 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:54.206274 21698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:54.206590 21700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:54.206622 21697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:54.207741 21688 server_base.cc:1061] running on GCE node
I20260809 07:51:54.208339 21688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:54.209477 21688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:54.210721 21688 hybrid_clock.cc:648] HybridClock initialized: now 1786261914210538 us; error 149 us; skew 500 ppm
I20260809 07:51:54.211056 21688 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:54 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261914, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:54.214644 21688 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:54.216320 21688 webserver.cc:533] Webserver started at http://127.26.37.1:33771/ using document root <none> and password file <none>
I20260809 07:51:54.217037 21688 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:54.217168 21688 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:54.222824 21688 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 07:51:54.225592 21710 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:54.226930 21688 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:51:54.227114 21688 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:51:54.227865 21688 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:54.249948 21688 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:54.253618 21688 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:54.253901 21688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:54.254827 21688 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:54.256488 21688 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:54.256572 21688 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:54.256728 21688 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:54.256830 21688 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:54.292428 21688 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34779
I20260809 07:51:54.292510 21856 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34779 every 8 connection(s)
I20260809 07:51:54.294028 21688 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:54.295349 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21688
Aug 09 07:51:54 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261914, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:51:54 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261914, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:51:54.332934 21332 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:51:54.334247 21594 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e12904b027a144d3a9245687086d3921" instance_seqno: 1786261912250225) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:40763; Asking this server to re-register.
I20260809 07:51:54.334841 21332 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:54.336153 21594 ts_manager.cc:194] Registered new tserver with Master: e12904b027a144d3a9245687086d3921 (127.26.37.2:44113)
Aug 09 07:51:54 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261914, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:51:54.339210 21858 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:51:54.339581 21859 heartbeater.cc:344] Connected to a master server at 127.26.37.61:40663
I20260809 07:51:54.339597 21858 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:54.339780 21859 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:54.340843 21858 heartbeater.cc:507] Master 127.26.37.62:41249 requested a full tablet report, sending...
I20260809 07:51:54.340864 21859 heartbeater.cc:507] Master 127.26.37.61:40663 requested a full tablet report, sending...
I20260809 07:51:54.342970 21594 ts_manager.cc:194] Registered new tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:34779)
I20260809 07:51:54.343243 20667 ts_manager.cc:194] Re-registered known tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:34779)
I20260809 07:51:54.344146 20667 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:39523
I20260809 07:51:54.345968 21857 heartbeater.cc:344] Connected to a master server at 127.26.37.60:42841
I20260809 07:51:54.346133 21857 heartbeater.cc:461] Registering TS with master...
I20260809 07:51:54.346623 21857 heartbeater.cc:507] Master 127.26.37.60:42841 requested a full tablet report, sending...
I20260809 07:51:54.347788 20785 ts_manager.cc:194] Re-registered known tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:34779)
I20260809 07:51:54.356549 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21688
W20260809 07:51:54.357833 21874 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:54.354379 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:55810 (local address 127.26.37.1:34779)
0809 07:51:54.355201 (+ 822us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:54.355207 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:54.355232 (+ 25us) server_negotiation.cc:408] Connection header received
0809 07:51:54.355275 (+ 43us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:54.355280 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:54.355453 (+ 173us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:54.355595 (+ 142us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:54.356483 (+ 888us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55810: BlockingRecv error: recv got EOF from 127.0.0.1:55810 (error 108)
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":502,"threads_started":1}
W20260809 07:51:54.359035 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21688: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:34779: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:54.359159 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21688
I20260809 07:51:54.386817 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34779
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--webserver_port=33771
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:54.743654 21876 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:54.744068 21876 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:54.744243 21876 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:51:54.744328 21876 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:54.753160 21876 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:54.753466 21876 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:54.764473 21876 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33771
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:54.766229 21876 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:54.767735 21876 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:54.778214 21883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:54.778167 21886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:54.778254 21884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:54.779356 21876 server_base.cc:1061] running on GCE node
I20260809 07:51:54.779960 21876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:54.781186 21876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:54.782444 21876 hybrid_clock.cc:648] HybridClock initialized: now 1786261914782264 us; error 131 us; skew 500 ppm
I20260809 07:51:54.782778 21876 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:54 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261914, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:54.786470 21876 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:54.788116 21876 webserver.cc:533] Webserver started at http://127.26.37.1:33771/ using document root <none> and password file <none>
I20260809 07:51:54.788964 21876 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:54.789080 21876 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:54.794485 21876 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:51:54.797480 21897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:54.798920 21876 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:51:54.799103 21876 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:51:54.799793 21876 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:54.821314 21876 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:54.825577 21876 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:54.825879 21876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:54.826960 21876 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:54.828639 21876 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:54.828747 21876 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:54.828903 21876 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:54.828976 21876 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:51:54.829442 21876 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:51:54.829532 21876 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:51:54.829614 21876 fault_injection.cc:59] injecting fault for kudu (pid 21876): FLAGS_fault_before_start (process will exit)
I20260809 07:51:54.856709 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34779
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--webserver_port=33771
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:55.213718 21913 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:55.214059 21913 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:55.214223 21913 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:55.222484 21913 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:55.222728 21913 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:51:55.232728 21913 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41955,127.26.37.21:37837,127.26.37.22:44811,127.26.37.23:46271,127.26.37.24:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33771
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:55.234351 21913 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:55.235769 21913 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:55.249902 21920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:55.251135 21921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:55.252791 21923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:55.253319 21913 server_base.cc:1061] running on GCE node
I20260809 07:51:55.254149 21913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:55.255795 21913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:55.257238 21913 hybrid_clock.cc:648] HybridClock initialized: now 1786261915257002 us; error 194 us; skew 500 ppm
I20260809 07:51:55.257719 21913 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:55 dist-test-slave-bw88 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261915, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:55.262450 21913 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:51:55.264206 21913 webserver.cc:533] Webserver started at http://127.26.37.1:33771/ using document root <none> and password file <none>
I20260809 07:51:55.264977 21913 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:55.265138 21913 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:55.271344 21913 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:51:55.274407 21933 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:55.275950 21913 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:51:55.276144 21913 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:51:55.276887 21913 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:55.297355 21913 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:55.301265 21913 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:55.301575 21913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:55.302650 21913 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:51:55.304396 21913 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:55.304526 21913 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:55.304659 21913 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:51:55.304754 21913 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:55.348639 21913 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34779
I20260809 07:51:55.348690 22085 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34779 every 8 connection(s)
I20260809 07:51:55.350260 21913 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:55.358291 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21913
I20260809 07:51:55.358719 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21913
Aug 09 07:51:55 dist-test-slave-bw88 krb5kdc[20487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261915, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
W20260809 07:51:55.392592 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21913: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:34779: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:55.392764 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21913
I20260809 07:51:55.429188 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21144
W20260809 07:51:55.429286 22101 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:55.381278 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:46305 (local address 127.26.37.62:41249)
0809 07:51:55.389262 (+ 7984us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:55.389270 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:55.389294 (+ 24us) server_negotiation.cc:408] Connection header received
0809 07:51:55.389403 (+ 109us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:55.389412 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:55.389511 (+ 99us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:55.389663 (+ 152us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:55.391259 (+ 1596us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:55.399443 (+ 8184us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:55.428906 (+ 29463us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:46305: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":7837,"thread_start_us":10084,"threads_started":1}
W20260809 07:51:55.431154 22097 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:55.355371 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:33187 (local address 127.26.37.60:42841)
0809 07:51:55.380064 (+ 24693us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:55.380070 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:55.380803 (+ 733us) server_negotiation.cc:408] Connection header received
0809 07:51:55.384419 (+ 3616us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:55.384430 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:55.384544 (+ 114us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:55.384664 (+ 120us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:55.387381 (+ 2717us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:55.391951 (+ 4570us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:55.430087 (+ 38136us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:33187: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":24544,"thread_start_us":24650,"threads_started":1}
W20260809 07:51:55.432696 22094 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:51:55.354285 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:53611 (local address 127.26.37.61:40663)
0809 07:51:55.354752 (+ 467us) server_negotiation.cc:207] Beginning negotiation
0809 07:51:55.354760 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:51:55.369142 (+ 14382us) server_negotiation.cc:408] Connection header received
0809 07:51:55.372101 (+ 2959us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:51:55.372111 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:51:55.372215 (+ 104us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:51:55.372346 (+ 131us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:51:55.376476 (+ 4130us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:55.378196 (+ 1720us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:55.380068 (+ 1872us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:55.380630 (+ 562us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:51:55.387696 (+ 7066us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0809 07:51:55.387742 (+ 46us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0809 07:51:55.387747 (+ 5us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
0809 07:51:55.387783 (+ 36us) server_negotiation.cc:923] calling sasl_server_start()
0809 07:51:55.391236 (+ 3453us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:55.392109 (+ 873us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
0809 07:51:55.392116 (+ 7us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
0809 07:51:55.392121 (+ 5us) server_negotiation.cc:968] calling sasl_server_step()
0809 07:51:55.392221 (+ 100us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:55.400166 (+ 7945us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
0809 07:51:55.400171 (+ 5us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
0809 07:51:55.400173 (+ 2us) server_negotiation.cc:968] calling sasl_server_step()
0809 07:51:55.400427 (+ 254us) server_negotiation.cc:378] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:55.400537 (+ 110us) server_negotiation.cc:1036] Waiting for connection context
0809 07:51:55.429085 (+ 28548us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:53611: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":603,"threads_started":1}
W20260809 07:51:55.433904 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21144: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:44113: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:55.434119 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21144
I20260809 07:51:55.455027 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21347
W20260809 07:51:55.457908 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21347: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:43219: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:55.458073 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21347
I20260809 07:51:55.482070 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21549
W20260809 07:51:55.484336 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21549: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:41249: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:51:55.484508 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21549
I20260809 07:51:55.516294 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20627
I20260809 07:51:55.517549 20716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:55.519186 20738 master_runner.cc:187] Error getting master registration for 127.26.37.62:41249: Network error: Client connection negotiation failed: client connection to 127.26.37.62:41249: connect: Connection refused (error 111),
I20260809 07:51:55.519447 20738 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:55.631093 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20627
I20260809 07:51:55.661095 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20738
I20260809 07:51:55.662528 20824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:51:55.771030 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20738
2026-08-09T07:51:55Z chronyd exiting
2026-08-09T07:51:55Z chronyd exiting
2026-08-09T07:51:55Z chronyd exiting
2026-08-09T07:51:55Z chronyd exiting
2026-08-09T07:51:55Z chronyd exiting
Aug 09 07:51:55 dist-test-slave-bw88 krb5kdc[22114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:55 dist-test-slave-bw88 krb5kdc[22114](info): set up 2 sockets
Aug 09 07:51:55 dist-test-slave-bw88 krb5kdc[22114](info): commencing operation
krb5kdc: starting...
W20260809 07:51:57.955765 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.099s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:51:58 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:51:58Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:58Z Disabled control of system clock
2026-08-09T07:51:58Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:58Z Disabled control of system clock
2026-08-09T07:51:58Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:58Z Disabled control of system clock
2026-08-09T07:51:58Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:58Z Disabled control of system clock
2026-08-09T07:51:58Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:58.368095 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41249
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:58.714169 22150 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:58.714524 22150 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:58.714695 22150 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:58.722852 22150 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:58.722963 22150 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:58.723101 22150 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:58.723193 22150 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:58.733618 22150 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:58.735206 22150 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:58.736630 22150 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:58.744835 22158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:58.744859 22159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:58.745519 22161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:58.746145 22150 server_base.cc:1061] running on GCE node
I20260809 07:51:58.746901 22150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:58.748311 22150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:58.749531 22150 hybrid_clock.cc:648] HybridClock initialized: now 1786261918749383 us; error 115 us; skew 500 ppm
I20260809 07:51:58.749903 22150 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:58 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261918, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:58.753238 22150 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:51:58.754846 22150 webserver.cc:533] Webserver started at http://127.26.37.62:37737/ using document root <none> and password file <none>
I20260809 07:51:58.755564 22150 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:58.755710 22150 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:58.761102 22150 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:58.764156 22173 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:58.765640 22150 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:58.765828 22150 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cd2b7bfe6a214907a7ee705b58948d46"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:58.766569 22150 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:58.799139 22150 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:58.806469 22150 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:58.806784 22150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:58.827970 22150 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41249
I20260809 07:51:58.827988 22239 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41249 every 8 connection(s)
I20260809 07:51:58.829685 22150 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:58.835462 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22150
I20260809 07:51:58.835724 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:51:58.837503 22240 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:58.842617 22240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap starting.
I20260809 07:51:58.861119 22240 log.cc:826] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:58.873006 22240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:58.873668 22240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap complete.
I20260809 07:51:58.880692 22240 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:58.881507 22240 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2b7bfe6a214907a7ee705b58948d46, State: Initialized, Role: FOLLOWER
I20260809 07:51:58.882256 22240 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:58.883174 22246 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:58.883487 22246 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:58.884323 22240 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:58.895588 22259 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:51:58.895686 22259 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: loading cluster ID for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:58.900216 22259 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: acquiring CA information for follower catalog manager: success
I20260809 07:51:58.902917 22259 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:58.917402 22150 master_runner.cc:187] Error getting master registration for 127.26.37.61:40663: Network error: Client connection negotiation failed: client connection to 127.26.37.61:40663: connect: Connection refused (error 111),
I20260809 07:51:58.920987 22150 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:58.968434 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:40663
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:59.321635 22269 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:59.321928 22269 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:59.322018 22269 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:59.329855 22269 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:59.329931 22269 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:59.329980 22269 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:59.330024 22269 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:59.339955 22269 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:59.341331 22269 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:59.342636 22269 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:59.350629 22275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:59.350672 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:59.351372 22279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:59.354559 22269 server_base.cc:1061] running on GCE node
I20260809 07:51:59.355249 22269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:59.356886 22269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:59.358170 22269 hybrid_clock.cc:648] HybridClock initialized: now 1786261919357986 us; error 150 us; skew 500 ppm
I20260809 07:51:59.358495 22269 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:59 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261919, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:59.361930 22269 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:51:59.363600 22269 webserver.cc:533] Webserver started at http://127.26.37.61:37033/ using document root <none> and password file <none>
I20260809 07:51:59.364303 22269 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:59.364455 22269 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:59.369801 22269 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:59.372695 22289 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:59.374042 22269 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:59.374212 22269 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "f8cd98f2b2c8473fbdcf497a37e26656"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:51:59.374899 22269 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:59.405463 22269 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:59.409031 22269 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:59.409304 22269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:59.430248 22269 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:40663
I20260809 07:51:59.430289 22363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:40663 every 8 connection(s)
I20260809 07:51:59.431743 22269 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:51:59.437703 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22269
I20260809 07:51:59.437901 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:51:59.438627 22364 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:59.441934 22364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: Bootstrap starting.
I20260809 07:51:59.457847 22364 log.cc:826] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:59.466305 22364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:59.466820 22364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: Bootstrap complete.
I20260809 07:51:59.473591 22364 raft_consensus.cc:359] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:59.474282 22364 raft_consensus.cc:740] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cd98f2b2c8473fbdcf497a37e26656, State: Initialized, Role: FOLLOWER
I20260809 07:51:59.475092 22364 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:51:59.475992 22372 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:51:59.476301 22372 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:59.477402 22364 sys_catalog.cc:565] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:59.488162 22387 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:51:59.488310 22387 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: loading cluster ID for follower catalog manager: success
I20260809 07:51:59.492614 22387 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: acquiring CA information for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:59.495507 22387 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:51:59 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261919, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:51:59.530469 22269 master_runner.cc:187] Error getting master registration for 127.26.37.60:42841: Network error: Client connection negotiation failed: client connection to 127.26.37.60:42841: connect: Connection refused (error 111),
I20260809 07:51:59.535202 22269 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:51:59.561529 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:42841
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:59.923048 22397 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:59.923385 22397 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:59.923569 22397 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:59.931753 22397 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:59.931867 22397 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:59.931937 22397 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:59.931983 22397 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:59.942451 22397 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:42841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:51:59.944017 22397 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:59.945655 22397 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:51:59.953915 22406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:59.953886 22407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:59.954396 22410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:59.955633 22397 server_base.cc:1061] running on GCE node
I20260809 07:51:59.956446 22397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:59.958050 22397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:59.959278 22397 hybrid_clock.cc:648] HybridClock initialized: now 1786261919959112 us; error 129 us; skew 500 ppm
I20260809 07:51:59.959673 22397 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:51:59 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261919, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:59.963193 22397 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:51:59.964645 22397 webserver.cc:533] Webserver started at http://127.26.37.60:35669/ using document root <none> and password file <none>
I20260809 07:51:59.965289 22397 fs_manager.cc:362] Metadata directory not provided
I20260809 07:51:59.965469 22397 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:59.970633 22397 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 07:51:59.973443 22427 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:59.974768 22397 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:51:59.974932 22397 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "4e853cca2d0243d7ba8334d9199b4815"
format_stamp: "Formatted at 2026-08-09 07:51:50 on dist-test-slave-bw88"
I20260809 07:51:59.975648 22397 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:59.992398 22397 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:51:59.996388 22397 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:59.996667 22397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:00.017642 22397 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:42841
I20260809 07:52:00.017659 22503 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:42841 every 8 connection(s)
I20260809 07:52:00.019344 22397 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:52:00.020798 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22397
I20260809 07:52:00.021025 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:52:00.028923 22504 sys_catalog.cc:263] Verifying existing consensus state
Aug 09 07:52:00 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:00.033922 22504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Bootstrap starting.
Aug 09 07:52:00 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:00.051230 22504 log.cc:826] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:00.060514 22504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:00.061045 22504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Bootstrap complete.
Aug 09 07:52:00 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:00.068018 22504 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:00.068753 22504 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e853cca2d0243d7ba8334d9199b4815, State: Initialized, Role: FOLLOWER
I20260809 07:52:00.069516 22504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:00.070453 22511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:00.070958 22511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:00.071874 22504 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:00.084784 22527 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:52:00.085803 22527 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: loading cluster ID for follower catalog manager: success
I20260809 07:52:00.090092 22527 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: acquiring CA information for follower catalog manager: success
Aug 09 07:52:00 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261919, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:00.093379 22527 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:52:00.119304 22397 master_runner.cc:264] No leader master found from master 4e853cca2d0243d7ba8334d9199b4815
I20260809 07:52:00.119470 22397 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:52:00.677917 22530 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:00.678273 22530 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:00.680845 22530 leader_election.cc:290] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8cd98f2b2c8473fbdcf497a37e26656 (127.26.37.61:40663), 4e853cca2d0243d7ba8334d9199b4815 (127.26.37.60:42841)
I20260809 07:52:00.688546 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd2b7bfe6a214907a7ee705b58948d46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" is_pre_election: true
I20260809 07:52:00.689067 22329 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd2b7bfe6a214907a7ee705b58948d46 in term 1.
I20260809 07:52:00.689743 22174 leader_election.cc:304] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd2b7bfe6a214907a7ee705b58948d46, f8cd98f2b2c8473fbdcf497a37e26656; no voters:
I20260809 07:52:00.690148 22530 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:52:00.690313 22530 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:00.690425 22530 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:52:00.691609 22468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd2b7bfe6a214907a7ee705b58948d46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e853cca2d0243d7ba8334d9199b4815" is_pre_election: true
I20260809 07:52:00.692091 22468 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd2b7bfe6a214907a7ee705b58948d46 in term 1.
I20260809 07:52:00.694613 22530 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:00.695197 22530 leader_election.cc:290] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [CANDIDATE]: Term 2 election: Requested vote from peers f8cd98f2b2c8473fbdcf497a37e26656 (127.26.37.61:40663), 4e853cca2d0243d7ba8334d9199b4815 (127.26.37.60:42841)
I20260809 07:52:00.695732 22468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd2b7bfe6a214907a7ee705b58948d46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e853cca2d0243d7ba8334d9199b4815"
I20260809 07:52:00.695734 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd2b7bfe6a214907a7ee705b58948d46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8cd98f2b2c8473fbdcf497a37e26656"
I20260809 07:52:00.695933 22329 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:52:00.695933 22468 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:52:00.701282 22468 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd2b7bfe6a214907a7ee705b58948d46 in term 2.
I20260809 07:52:00.701283 22329 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd2b7bfe6a214907a7ee705b58948d46 in term 2.
I20260809 07:52:00.701885 22176 leader_election.cc:304] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e853cca2d0243d7ba8334d9199b4815, cd2b7bfe6a214907a7ee705b58948d46; no voters:
I20260809 07:52:00.702221 22530 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:52:00.702617 22530 raft_consensus.cc:697] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 2 LEADER]: Becoming Leader. State: Replica: cd2b7bfe6a214907a7ee705b58948d46, State: Running, Role: LEADER
I20260809 07:52:00.703107 22530 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:00.705744 22538 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd2b7bfe6a214907a7ee705b58948d46. Latest consensus state: current_term: 2 leader_uuid: "cd2b7bfe6a214907a7ee705b58948d46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:00.706094 22538 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:00.706806 22541 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:52:00.708822 22541 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:52:00.709717 22541 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:52:00.709815 22541 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:52:00.710930 22541 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:52:00.711802 22541 catalog_manager.cc:6103] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Loaded TSK: 0
I20260809 07:52:00.713575 22541 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:00.858485 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:01.200152 22546 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:01.200456 22546 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:01.200656 22546 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:01.208962 22546 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:01.209177 22546 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:01.219579 22546 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:01.221267 22546 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:01.222788 22546 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:01.233304 22553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:01.233395 22554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:01.234077 22556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:01.234699 22546 server_base.cc:1061] running on GCE node
I20260809 07:52:01.235204 22546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:01.236229 22546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:01.237438 22546 hybrid_clock.cc:648] HybridClock initialized: now 1786261921237294 us; error 106 us; skew 500 ppm
I20260809 07:52:01.237763 22546 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:01 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261921, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:01.241397 22546 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:01.243289 22546 webserver.cc:533] Webserver started at http://127.26.37.1:35649/ using document root <none> and password file <none>
I20260809 07:52:01.243718 22468 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 2 FOLLOWER]: Refusing update from remote peer cd2b7bfe6a214907a7ee705b58948d46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:52:01.244081 22546 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:01.244212 22546 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:01.244711 22559 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:52:01.248726 22563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd2b7bfe6a214907a7ee705b58948d46. Latest consensus state: current_term: 2 leader_uuid: "cd2b7bfe6a214907a7ee705b58948d46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:01.249044 22563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:01.250662 22559 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "cd2b7bfe6a214907a7ee705b58948d46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:01.250962 22559 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:01.251596 22546 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 07:52:01.252807 22329 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 2 FOLLOWER]: Refusing update from remote peer cd2b7bfe6a214907a7ee705b58948d46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:52:01.256942 22569 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:52:01.263679 22563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "cd2b7bfe6a214907a7ee705b58948d46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:01.263960 22563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:01.265952 22576 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:01.267979 22546 fs_manager.cc:730] Time spent opening block manager: real 0.014s user 0.009s sys 0.003s
I20260809 07:52:01.268240 22546 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:52:01.269006 22582 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "cd2b7bfe6a214907a7ee705b58948d46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:01.269295 22582 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:01.269419 22546 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:01.269850 22578 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd2b7bfe6a214907a7ee705b58948d46. Latest consensus state: current_term: 2 leader_uuid: "cd2b7bfe6a214907a7ee705b58948d46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:01.270094 22578 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:01.290729 22546 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:01.294819 22546 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:01.295104 22546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:01.296092 22546 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:01.297763 22546 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:01.297873 22546 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:01.298007 22546 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:01.298106 22546 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:01.332319 22546 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:41125
I20260809 07:52:01.332409 22734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:41125 every 8 connection(s)
I20260809 07:52:01.333945 22546 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:01.343254 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22546
I20260809 07:52:01.343478 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:52:01 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261921, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:01 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261921, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:01 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261921, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:01.385363 22737 heartbeater.cc:344] Connected to a master server at 127.26.37.60:42841
I20260809 07:52:01.385417 22739 heartbeater.cc:344] Connected to a master server at 127.26.37.61:40663
I20260809 07:52:01.385515 22738 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:52:01.385869 22738 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:01.385861 22739 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:01.385861 22737 heartbeater.cc:461] Registering TS with master...
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:52:01.387131 22737 heartbeater.cc:507] Master 127.26.37.60:42841 requested a full tablet report, sending...
I20260809 07:52:01.387154 22738 heartbeater.cc:507] Master 127.26.37.62:41249 requested a full tablet report, sending...
I20260809 07:52:01.387555 22739 heartbeater.cc:507] Master 127.26.37.61:40663 requested a full tablet report, sending...
I20260809 07:52:01.389840 22318 ts_manager.cc:194] Registered new tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:41125)
I20260809 07:52:01.389839 22198 ts_manager.cc:194] Registered new tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:41125)
I20260809 07:52:01.390636 22453 ts_manager.cc:194] Registered new tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:41125)
I20260809 07:52:01.392012 22198 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:35467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:01.468597 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:01.747291 22559 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "cd2b7bfe6a214907a7ee705b58948d46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:01.747574 22559 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: LEADER
W20260809 07:52:01.819314 22754 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:01.819669 22754 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:01.819841 22754 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:01.828585 22754 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:01.828795 22754 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:52:01.839670 22754 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:01.841275 22754 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:01.842653 22754 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:01.851895 22764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:01.851922 22763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:01.852703 22767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:01.853231 22754 server_base.cc:1061] running on GCE node
I20260809 07:52:01.853798 22754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:01.854964 22754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:01.856168 22754 hybrid_clock.cc:648] HybridClock initialized: now 1786261921856047 us; error 89 us; skew 500 ppm
I20260809 07:52:01.856459 22754 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:01 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261921, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:01.860035 22754 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:52:01.861800 22754 webserver.cc:533] Webserver started at http://127.26.37.2:36189/ using document root <none> and password file <none>
I20260809 07:52:01.862532 22754 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:01.862672 22754 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:01.867938 22754 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 07:52:01.870708 22776 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:01.872126 22754 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:01.872303 22754 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "e12904b027a144d3a9245687086d3921"
format_stamp: "Formatted at 2026-08-09 07:51:52 on dist-test-slave-bw88"
I20260809 07:52:01.872983 22754 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:01.907996 22754 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:01.912084 22754 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:01.912319 22754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:01.913244 22754 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:01.914809 22754 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:01.914899 22754 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:01.915009 22754 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:01.915067 22754 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:01.950682 22754 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:38163
I20260809 07:52:01.950767 22931 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:38163 every 8 connection(s)
I20260809 07:52:01.952288 22754 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:01.956349 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22754
I20260809 07:52:01.956599 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:52:01 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261921, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:01 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261921, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:01 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261921, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:01.984040 22932 heartbeater.cc:344] Connected to a master server at 127.26.37.60:42841
I20260809 07:52:01.984225 22934 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:52:01.984463 22936 heartbeater.cc:344] Connected to a master server at 127.26.37.61:40663
I20260809 07:52:01.984560 22932 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:01.984563 22934 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:01.984623 22936 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:01.985868 22932 heartbeater.cc:507] Master 127.26.37.60:42841 requested a full tablet report, sending...
I20260809 07:52:01.985909 22934 heartbeater.cc:507] Master 127.26.37.62:41249 requested a full tablet report, sending...
I20260809 07:52:01.985939 22936 heartbeater.cc:507] Master 127.26.37.61:40663 requested a full tablet report, sending...
I20260809 07:52:01.987885 22198 ts_manager.cc:194] Registered new tserver with Master: e12904b027a144d3a9245687086d3921 (127.26.37.2:38163)
I20260809 07:52:01.988058 22453 ts_manager.cc:194] Registered new tserver with Master: e12904b027a144d3a9245687086d3921 (127.26.37.2:38163)
I20260809 07:52:01.989037 22198 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:54381
I20260809 07:52:01.989945 22318 ts_manager.cc:194] Registered new tserver with Master: e12904b027a144d3a9245687086d3921 (127.26.37.2:38163)
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:02.085716 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:02.394552 22738 heartbeater.cc:499] Master 127.26.37.62:41249 was elected leader, sending a full tablet report...
W20260809 07:52:02.447614 22951 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:02.447901 22951 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:02.448071 22951 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:02.455893 22951 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:02.456130 22951 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:52:02.466645 22951 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:02.468313 22951 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:02.469791 22951 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:02.480269 22956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:02.480805 22957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:02.484359 22951 server_base.cc:1061] running on GCE node
W20260809 07:52:02.484958 22960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:02.485749 22951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:02.486951 22951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:02.488303 22951 hybrid_clock.cc:648] HybridClock initialized: now 1786261922488147 us; error 124 us; skew 500 ppm
I20260809 07:52:02.488644 22951 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:02 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261922, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:02.492321 22951 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:52:02.494091 22951 webserver.cc:533] Webserver started at http://127.26.37.3:33507/ using document root <none> and password file <none>
I20260809 07:52:02.494864 22951 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:02.495020 22951 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:02.500836 22951 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 07:52:02.504007 22968 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:02.505539 22951 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:02.505713 22951 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "8cb20c1bb15e433c9cd550b59ce25b03"
format_stamp: "Formatted at 2026-08-09 07:51:52 on dist-test-slave-bw88"
I20260809 07:52:02.506497 22951 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:02.520525 22951 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:02.524609 22951 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:02.524896 22951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:02.525979 22951 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:02.527612 22951 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:02.527735 22951 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:02.527877 22951 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:02.527971 22951 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:02.564020 22951 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:44299
I20260809 07:52:02.564110 23115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:44299 every 8 connection(s)
I20260809 07:52:02.565733 22951 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:02.574193 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22951
I20260809 07:52:02.574427 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:52:02 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261922, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:02 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261922, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:02 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261922, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:02.595297 23118 heartbeater.cc:344] Connected to a master server at 127.26.37.61:40663
I20260809 07:52:02.595377 23117 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:52:02.595573 23116 heartbeater.cc:344] Connected to a master server at 127.26.37.60:42841
I20260809 07:52:02.595804 23116 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:02.595814 23117 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:02.596267 23118 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:02.597489 23118 heartbeater.cc:507] Master 127.26.37.61:40663 requested a full tablet report, sending...
I20260809 07:52:02.597536 23116 heartbeater.cc:507] Master 127.26.37.60:42841 requested a full tablet report, sending...
I20260809 07:52:02.597568 23117 heartbeater.cc:507] Master 127.26.37.62:41249 requested a full tablet report, sending...
I20260809 07:52:02.599596 22198 ts_manager.cc:194] Registered new tserver with Master: 8cb20c1bb15e433c9cd550b59ce25b03 (127.26.37.3:44299)
I20260809 07:52:02.599718 22453 ts_manager.cc:194] Registered new tserver with Master: 8cb20c1bb15e433c9cd550b59ce25b03 (127.26.37.3:44299)
I20260809 07:52:02.600668 22198 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:52087
I20260809 07:52:02.601992 22314 ts_manager.cc:194] Registered new tserver with Master: 8cb20c1bb15e433c9cd550b59ce25b03 (127.26.37.3:44299)
I20260809 07:52:02.617183 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:52:02.625010 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:52:02.625178 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22150
I20260809 07:52:02.626082 22235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:02.735967 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22150
I20260809 07:52:02.767342 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41249
--webserver_interface=127.26.37.62
--webserver_port=37737
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:02.991463 22934 heartbeater.cc:499] Master 127.26.37.62:41249 was elected leader, sending a full tablet report...
W20260809 07:52:02.993216 22934 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:41249 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:41249: connect: Connection refused (error 111)
W20260809 07:52:03.160808 23128 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:03.161134 23128 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:03.161249 23128 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:03.169685 23128 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:03.169788 23128 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:03.169844 23128 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:03.169896 23128 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:03.181629 23128 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=37737
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:03.183342 23128 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:03.184713 23128 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:03.193930 23137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:03.193981 23135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:03.194043 23139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:03.194553 23128 server_base.cc:1061] running on GCE node
I20260809 07:52:03.195338 23128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:03.196955 23128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:03.198280 23128 hybrid_clock.cc:648] HybridClock initialized: now 1786261923198101 us; error 131 us; skew 500 ppm
I20260809 07:52:03.198771 23128 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:03 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261923, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:03.202476 23128 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:52:03.204116 23128 webserver.cc:533] Webserver started at http://127.26.37.62:37737/ using document root <none> and password file <none>
I20260809 07:52:03.204840 23128 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:03.204990 23128 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:03.210625 23128 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:52:03.213672 23147 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:03.215157 23128 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:03.215351 23128 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cd2b7bfe6a214907a7ee705b58948d46"
format_stamp: "Formatted at 2026-08-09 07:51:49 on dist-test-slave-bw88"
I20260809 07:52:03.216104 23128 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:03.229174 23128 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:03.233121 23128 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:03.233462 23128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:03.255103 23128 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41249
I20260809 07:52:03.255122 23219 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41249 every 8 connection(s)
I20260809 07:52:03.256740 23128 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:03.263788 23220 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:52:03.263787 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 23128
I20260809 07:52:03.264163 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22546
I20260809 07:52:03.268163 23220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap starting.
Aug 09 07:52:03 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:52:03.283725 22730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:03.287667 23220 log.cc:826] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:03.299623 23220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:03.300372 23220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: Bootstrap complete.
I20260809 07:52:03.310626 23220 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:03.311550 23220 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2b7bfe6a214907a7ee705b58948d46, State: Initialized, Role: FOLLOWER
I20260809 07:52:03.312482 23220 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:03.313314 23226 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:03.313657 23226 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:03.314565 23220 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:03.336952 23244 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:52:03.337096 23244 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: loading cluster ID for follower catalog manager: success
I20260809 07:52:03.341121 23244 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: acquiring CA information for follower catalog manager: success
I20260809 07:52:03.343689 23244 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:52:03 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261923, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:03.388620 23128 master_runner.cc:264] No leader master found from master cd2b7bfe6a214907a7ee705b58948d46
I20260809 07:52:03.388789 23128 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:52:03.401628 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22546
W20260809 07:52:03.425732 22430 connection.cc:573] server connection from 127.26.37.1:33059 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:52:03.427039 23247 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:03.401066 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:53907 (local address 127.26.37.62:41249)
0809 07:52:03.401251 (+ 185us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:03.401258 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:03.425706 (+ 24448us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:53907: BlockingRecv error: recv got EOF from 127.26.37.1:53907 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 07:52:03.427109 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:41125
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--webserver_port=35649
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:03.603853 23117 heartbeater.cc:499] Master 127.26.37.62:41249 was elected leader, sending a full tablet report...
I20260809 07:52:03.616703 23169 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8cb20c1bb15e433c9cd550b59ce25b03" instance_seqno: 1786261922555319) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:47507; Asking this server to re-register.
I20260809 07:52:03.617842 23117 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:03.619938 23169 ts_manager.cc:194] Registered new tserver with Master: 8cb20c1bb15e433c9cd550b59ce25b03 (127.26.37.3:44299)
W20260809 07:52:03.803110 23252 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:03.803481 23252 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:03.803653 23252 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:03.812497 23252 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:03.812747 23252 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:03.823491 23252 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:41125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=35649
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:03.825120 23252 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:03.826623 23252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:03.837105 23262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:03.837064 23261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:03.838156 23265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:03.838451 23252 server_base.cc:1061] running on GCE node
I20260809 07:52:03.839078 23252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:03.840184 23252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:03.841543 23252 hybrid_clock.cc:648] HybridClock initialized: now 1786261923841369 us; error 129 us; skew 500 ppm
I20260809 07:52:03.841813 23252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:03 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261923, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:03.845542 23252 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:03.847088 23252 webserver.cc:533] Webserver started at http://127.26.37.1:35649/ using document root <none> and password file <none>
I20260809 07:52:03.847801 23252 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:03.847952 23252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:03.853305 23252 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:03.856091 23272 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:03.857628 23252 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:03.857815 23252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:52:03.858534 23252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:03.877202 23252 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:03.886965 23252 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:03.887271 23252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:03.888368 23252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:03.890086 23252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:03.890244 23252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:03.890388 23252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:03.890489 23252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:03.895695 23284 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd2b7bfe6a214907a7ee705b58948d46)
I20260809 07:52:03.896054 23284 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:03.899423 23284 leader_election.cc:290] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd2b7bfe6a214907a7ee705b58948d46 (127.26.37.62:41249), f8cd98f2b2c8473fbdcf497a37e26656 (127.26.37.61:40663)
I20260809 07:52:03.899507 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e853cca2d0243d7ba8334d9199b4815" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" is_pre_election: true
I20260809 07:52:03.907668 23182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e853cca2d0243d7ba8334d9199b4815" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd2b7bfe6a214907a7ee705b58948d46" is_pre_election: true
I20260809 07:52:03.908320 23182 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e853cca2d0243d7ba8334d9199b4815 in term 2.
I20260809 07:52:03.909132 22432 leader_election.cc:304] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e853cca2d0243d7ba8334d9199b4815, cd2b7bfe6a214907a7ee705b58948d46; no voters: f8cd98f2b2c8473fbdcf497a37e26656
I20260809 07:52:03.910023 23284 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:52:03.910152 23284 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd2b7bfe6a214907a7ee705b58948d46)
I20260809 07:52:03.910259 23284 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:52:03.914065 23284 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:03.915068 23284 leader_election.cc:290] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [CANDIDATE]: Term 3 election: Requested vote from peers cd2b7bfe6a214907a7ee705b58948d46 (127.26.37.62:41249), f8cd98f2b2c8473fbdcf497a37e26656 (127.26.37.61:40663)
I20260809 07:52:03.916114 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e853cca2d0243d7ba8334d9199b4815" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f8cd98f2b2c8473fbdcf497a37e26656"
I20260809 07:52:03.916163 23182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e853cca2d0243d7ba8334d9199b4815" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd2b7bfe6a214907a7ee705b58948d46"
I20260809 07:52:03.916440 23182 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:52:03.922880 23182 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e853cca2d0243d7ba8334d9199b4815 in term 3.
I20260809 07:52:03.923337 22432 leader_election.cc:304] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e853cca2d0243d7ba8334d9199b4815, cd2b7bfe6a214907a7ee705b58948d46; no voters: f8cd98f2b2c8473fbdcf497a37e26656
I20260809 07:52:03.923786 23284 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:52:03.924307 23284 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [term 3 LEADER]: Becoming Leader. State: Replica: 4e853cca2d0243d7ba8334d9199b4815, State: Running, Role: LEADER
I20260809 07:52:03.924852 23284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:03.927181 23385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e853cca2d0243d7ba8334d9199b4815. Latest consensus state: current_term: 3 leader_uuid: "4e853cca2d0243d7ba8334d9199b4815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:03.927508 23385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:03.929217 23396 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:52:03.932211 23396 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:52:03.933308 23396 catalog_manager.cc:1295] Loaded cluster ID: df717c7d674a48daba924d86568c7731
I20260809 07:52:03.933472 23396 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:52:03.934309 23252 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:41125
I20260809 07:52:03.934358 23422 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:41125 every 8 connection(s)
I20260809 07:52:03.935015 23396 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:52:03.936038 23396 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815: Loaded TSK: 0
I20260809 07:52:03.936280 23252 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:03.937265 23396 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:52:03.942354 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 23252
Aug 09 07:52:03 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261923, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:03 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261923, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:03 dist-test-slave-bw88 krb5kdc[22114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261923, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:03.974231 23426 heartbeater.cc:344] Connected to a master server at 127.26.37.61:40663
I20260809 07:52:03.974318 23424 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:52:03.974648 23424 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:03.974653 23426 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:03.975895 23426 heartbeater.cc:507] Master 127.26.37.61:40663 requested a full tablet report, sending...
I20260809 07:52:03.975919 23424 heartbeater.cc:507] Master 127.26.37.62:41249 requested a full tablet report, sending...
I20260809 07:52:03.978474 22314 ts_manager.cc:194] Re-registered known tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:41125)
I20260809 07:52:03.979302 23169 ts_manager.cc:194] Registered new tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:41125)
I20260809 07:52:03.980726 23423 heartbeater.cc:344] Connected to a master server at 127.26.37.60:42841
I20260809 07:52:03.980906 23423 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:03.981467 23423 heartbeater.cc:507] Master 127.26.37.60:42841 requested a full tablet report, sending...
I20260809 07:52:03.982730 22453 ts_manager.cc:194] Re-registered known tserver with Master: 11ffc5f5d7cf492d89c9adbff379827f (127.26.37.1:41125)
I20260809 07:52:03.985002 22453 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:35825
I20260809 07:52:04.000921 22934 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41249
I20260809 07:52:04.001183 22934 heartbeater.cc:499] Master 127.26.37.62:41249 was elected leader, sending a full tablet report...
I20260809 07:52:04.002470 23169 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e12904b027a144d3a9245687086d3921" instance_seqno: 1786261921942114) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:40727; Asking this server to re-register.
I20260809 07:52:04.003197 22934 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:04.004408 23169 ts_manager.cc:194] Registered new tserver with Master: e12904b027a144d3a9245687086d3921 (127.26.37.2:38163)
I20260809 07:52:04.004824 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23252
W20260809 07:52:04.006134 23444 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:04.002189 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:53980 (local address 127.26.37.1:41125)
0809 07:52:04.002963 (+ 774us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:04.002972 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:04.003003 (+ 31us) server_negotiation.cc:408] Connection header received
0809 07:52:04.003062 (+ 59us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:04.003069 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:04.003289 (+ 220us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:04.003517 (+ 228us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:04.004673 (+ 1156us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53980: BlockingRecv error: recv got EOF from 127.0.0.1:53980 (error 108)
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":434,"threads_started":1}
W20260809 07:52:04.007441 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23252: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:41125: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:04.007612 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 23252
W20260809 07:52:04.007635 23444 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:04.005712 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:53992 (local address 127.26.37.1:41125)
0809 07:52:04.006456 (+ 744us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:04.006463 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:04.006484 (+ 21us) server_negotiation.cc:408] Connection header received
0809 07:52:04.006539 (+ 55us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:04.006545 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:04.006632 (+ 87us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:04.006763 (+ 131us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:04.007355 (+ 592us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53992: BlockingRecv error: recv got EOF from 127.0.0.1:53992 (error 108)
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":1028,"threads_started":1}
I20260809 07:52:04.033981 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:41125
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--webserver_port=35649
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:04.117929 23450 raft_consensus.cc:493] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd2b7bfe6a214907a7ee705b58948d46)
I20260809 07:52:04.118235 23450 raft_consensus.cc:515] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } }
I20260809 07:52:04.120211 23450 leader_election.cc:290] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd2b7bfe6a214907a7ee705b58948d46 (127.26.37.62:41249), 4e853cca2d0243d7ba8334d9199b4815 (127.26.37.60:42841)
I20260809 07:52:04.128083 23182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd2b7bfe6a214907a7ee705b58948d46" is_pre_election: true
I20260809 07:52:04.128496 23182 raft_consensus.cc:2393] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f8cd98f2b2c8473fbdcf497a37e26656 in current term 3: Already voted for candidate 4e853cca2d0243d7ba8334d9199b4815 in this term.
I20260809 07:52:04.128937 22468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e853cca2d0243d7ba8334d9199b4815" is_pre_election: true
I20260809 07:52:04.129652 22294 leader_election.cc:304] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8cd98f2b2c8473fbdcf497a37e26656; no voters: 4e853cca2d0243d7ba8334d9199b4815, cd2b7bfe6a214907a7ee705b58948d46
I20260809 07:52:04.130028 23450 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:52:04.132129 23450 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:52:04.361892 22329 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [term 3 FOLLOWER]: Refusing update from remote peer 4e853cca2d0243d7ba8334d9199b4815: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:52:04.362723 23385 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:52:04.364944 23450 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e853cca2d0243d7ba8334d9199b4815. Latest consensus state: current_term: 3 leader_uuid: "4e853cca2d0243d7ba8334d9199b4815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:04.365240 23450 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:04.367113 23284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "4e853cca2d0243d7ba8334d9199b4815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:04.367404 23284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:04.368283 23182 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [term 3 FOLLOWER]: Refusing update from remote peer 4e853cca2d0243d7ba8334d9199b4815: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:52:04.368350 23450 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "4e853cca2d0243d7ba8334d9199b4815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:04.368551 23450 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cd98f2b2c8473fbdcf497a37e26656 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:04.369063 23385 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:52:04.373427 23460 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "4e853cca2d0243d7ba8334d9199b4815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:04.373776 23460 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:04.374197 23457 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e853cca2d0243d7ba8334d9199b4815. Latest consensus state: current_term: 3 leader_uuid: "4e853cca2d0243d7ba8334d9199b4815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:04.374418 23457 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2b7bfe6a214907a7ee705b58948d46 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:52:04.393957 23447 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:04.394275 23447 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:04.394459 23447 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:52:04.394546 23447 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:04.402863 23447 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:04.403123 23447 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:04.413719 23447 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:41125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=35649
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:04.415271 23447 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:04.416685 23447 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:04.427037 23469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:04.427006 23472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:04.426921 23470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:04.428455 23447 server_base.cc:1061] running on GCE node
I20260809 07:52:04.429054 23447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:04.430189 23447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:04.431537 23447 hybrid_clock.cc:648] HybridClock initialized: now 1786261924431447 us; error 30 us; skew 500 ppm
I20260809 07:52:04.431859 23447 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:04 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261924, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:04.436208 23447 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:04.443238 23447 webserver.cc:533] Webserver started at http://127.26.37.1:35649/ using document root <none> and password file <none>
I20260809 07:52:04.444022 23447 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:04.444136 23447 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:04.449641 23447 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 07:52:04.452370 23479 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:04.453744 23447 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:04.453917 23447 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:52:04.454595 23447 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:04.489164 23447 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:04.492849 23447 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:04.493167 23447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:04.494242 23447 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:04.495841 23447 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:04.495914 23447 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:04.496114 23447 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:04.496168 23447 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:52:04.496580 23447 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:52:04.496650 23447 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:52:04.496735 23447 fault_injection.cc:59] injecting fault for kudu (pid 23447): FLAGS_fault_before_start (process will exit)
I20260809 07:52:04.523782 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:41125
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--webserver_port=35649
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:04.889490 23491 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:04.889822 23491 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:04.889918 23491 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:04.898113 23491 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:04.898293 23491 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:04.909021 23491 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43541,127.26.37.21:40869,127.26.37.22:40781,127.26.37.23:40251,127.26.37.24:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:41125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=35649
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41249,127.26.37.61:40663,127.26.37.60:42841
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:04.910665 23491 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:04.912082 23491 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:04.922569 23498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:04.922670 23497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:04.923823 23501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:04.924401 23491 server_base.cc:1061] running on GCE node
I20260809 07:52:04.924930 23491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:04.926103 23491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:04.927335 23491 hybrid_clock.cc:648] HybridClock initialized: now 1786261924927194 us; error 106 us; skew 500 ppm
I20260809 07:52:04.927666 23491 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:04 dist-test-slave-bw88 krb5kdc[22114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261924, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:04.931831 23491 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:04.933815 23491 webserver.cc:533] Webserver started at http://127.26.37.1:35649/ using document root <none> and password file <none>
I20260809 07:52:04.934615 23491 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:04.934752 23491 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:04.940276 23491 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 07:52:04.943091 23509 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:04.944531 23491 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:52:04.944710 23491 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "11ffc5f5d7cf492d89c9adbff379827f"
format_stamp: "Formatted at 2026-08-09 07:51:51 on dist-test-slave-bw88"
I20260809 07:52:04.945515 23491 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:04.967314 23491 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:04.970947 23491 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:04.971235 23491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:04.972246 23491 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:04.973929 23491 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:04.974066 23491 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:04.974197 23491 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:04.974287 23491 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:04.978904 23495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "4e853cca2d0243d7ba8334d9199b4815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2b7bfe6a214907a7ee705b58948d46" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41249 } } peers { permanent_uuid: "f8cd98f2b2c8473fbdcf497a37e26656" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 40663 } } peers { permanent_uuid: "4e853cca2d0243d7ba8334d9199b4815" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 42841 } } }
I20260809 07:52:04.979219 23495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e853cca2d0243d7ba8334d9199b4815 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:04.993486 22932 heartbeater.cc:499] Master 127.26.37.60:42841 was elected leader, sending a full tablet report...
I20260809 07:52:05.013548 23491 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:41125
I20260809 07:52:05.013645 23665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:41125 every 8 connection(s)
I20260809 07:52:05.015089 23491 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:05.015954 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 23491
I20260809 07:52:05.016361 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23491
W20260809 07:52:05.044701 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23491: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:41125: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:05.044878 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 23491
I20260809 07:52:05.070108 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22754
W20260809 07:52:05.070243 23676 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:05.030884 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:42955 (local address 127.26.37.62:41249)
0809 07:52:05.035089 (+ 4205us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:05.035098 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:05.038639 (+ 3541us) server_negotiation.cc:408] Connection header received
0809 07:52:05.039338 (+ 699us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:05.039345 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:05.039422 (+ 77us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:05.039508 (+ 86us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:05.044263 (+ 4755us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:05.045805 (+ 1542us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:05.069820 (+ 24015us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:42955: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":4029,"thread_start_us":3910,"threads_started":1}
W20260809 07:52:05.071090 23680 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:05.036876 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:43887 (local address 127.26.37.61:40663)
0809 07:52:05.049225 (+ 12349us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:05.049231 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:05.069681 (+ 20450us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:43887: BlockingRecv error: recv got EOF from 127.26.37.1:43887 (error 108)
Metrics: {"server-negotiator.queue_time_us":12219,"thread_start_us":12107,"threads_started":1}
W20260809 07:52:05.071399 23671 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:05.018464 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:53771 (local address 127.26.37.60:42841)
0809 07:52:05.033985 (+ 15521us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:05.033994 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:05.034024 (+ 30us) server_negotiation.cc:408] Connection header received
0809 07:52:05.034104 (+ 80us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:05.034112 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:05.034216 (+ 104us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:05.034371 (+ 155us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:05.042462 (+ 8091us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:05.043914 (+ 1452us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:05.069885 (+ 25971us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:53771: BlockingRecv error: recv got EOF from 127.26.37.1:53771 (error 108)
Metrics: {"server-negotiator.queue_time_us":15348,"thread_start_us":16087,"threads_started":1}
W20260809 07:52:05.073257 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22754: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:38163: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:05.073421 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22754
I20260809 07:52:05.099156 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22951
W20260809 07:52:05.102080 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22951: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:44299: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:05.102289 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22951
I20260809 07:52:05.128047 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23128
W20260809 07:52:05.130163 23676 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:05.128802 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.26.37.62:41249)
0809 07:52:05.128957 (+ 155us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:05.128961 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:05.129017 (+ 56us) server_negotiation.cc:408] Connection header received
0809 07:52:05.129183 (+ 166us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:05.129188 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:05.129250 (+ 62us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:05.129410 (+ 160us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:05.129988 (+ 578us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46404: BlockingRecv error: recv got EOF from 127.0.0.1:46404 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 07:52:05.130160 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23128: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:41249: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:05.130309 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 23128
I20260809 07:52:05.157063 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22269
I20260809 07:52:05.158284 22359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:05.268158 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22269
I20260809 07:52:05.297559 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22397
I20260809 07:52:05.298799 22499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:05.408358 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22397
2026-08-09T07:52:05Z chronyd exiting
2026-08-09T07:52:05Z chronyd exiting
2026-08-09T07:52:05Z chronyd exiting
2026-08-09T07:52:05Z chronyd exiting
2026-08-09T07:52:05Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/8 (19308 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/9
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:05 dist-test-slave-bw88 krb5kdc[23688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:05 dist-test-slave-bw88 krb5kdc[23688](info): set up 2 sockets
Aug 09 07:52:05 dist-test-slave-bw88 krb5kdc[23688](info): commencing operation
krb5kdc: starting...
W20260809 07:52:07.661820 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.158s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:52:07 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:52:15: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:52:17 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261937, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:52:23 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261943, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:52:23 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261943, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:24.203037 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33717
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:24.553546 24095 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:24.553928 24095 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:24.554108 24095 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:24.562438 24095 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:24.562551 24095 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:24.562649 24095 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:24.562742 24095 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:24.573606 24095 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:24.575286 24095 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:24.576714 24095 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:24.585578 24104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:24.585726 24111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:24.586344 24101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:24.586318 24095 server_base.cc:1061] running on GCE node
I20260809 07:52:24.587200 24095 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:24.587339 24095 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:24.587404 24095 hybrid_clock.cc:648] HybridClock initialized: now 1786261944587386 us; error 0 us; skew 500 ppm
I20260809 07:52:24.587734 24095 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:24 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:24.591099 24095 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:52:24.593031 24095 webserver.cc:533] Webserver started at http://127.26.37.62:38147/ using document root <none> and password file <none>
I20260809 07:52:24.593851 24095 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:24.593997 24095 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:24.594350 24095 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:24.596701 24095 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "74e05caec0c443b1bfbd167de9c9f8da"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-bw88"
I20260809 07:52:24.597561 24095 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "74e05caec0c443b1bfbd167de9c9f8da"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-bw88"
I20260809 07:52:24.602846 24095 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 07:52:24.606320 24119 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:24.607591 24095 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:52:24.607762 24095 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "74e05caec0c443b1bfbd167de9c9f8da"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-bw88"
I20260809 07:52:24.607980 24095 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:24.630463 24095 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:24.634720 24095 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:24.635047 24095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:24.656474 24095 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33717
I20260809 07:52:24.656522 24189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33717 every 8 connection(s)
I20260809 07:52:24.658186 24095 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:24.660861 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24095
I20260809 07:52:24.661089 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:52:24.662988 24190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:24.669566 24190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:52:24 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
I20260809 07:52:24.696187 24190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
W20260809 07:52:24.697590 24122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34039: connect: Connection refused (error 111)
W20260809 07:52:24.699311 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34039: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34039: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:24.748771 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } attempt: 1
W20260809 07:52:24.751055 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34039: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34039: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:24.795619 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:34039
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:24.819361 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } attempt: 2
W20260809 07:52:24.821638 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34039: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34039: connect: Connection refused (error 111)
I20260809 07:52:24.912942 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } attempt: 3
W20260809 07:52:24.915292 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34039: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34039: connect: Connection refused (error 111)
I20260809 07:52:25.058596 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } attempt: 4
W20260809 07:52:25.060834 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34039: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34039: connect: Connection refused (error 111)
W20260809 07:52:25.150295 24202 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:25.150758 24202 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:25.150928 24202 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:25.159441 24202 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:25.159590 24202 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:25.159716 24202 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:25.159801 24202 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:25.170730 24202 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:34039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:25.172447 24202 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:25.174152 24202 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:25.182762 24208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.182717 24209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.183801 24211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:25.184290 24202 server_base.cc:1061] running on GCE node
I20260809 07:52:25.185011 24202 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:25.185185 24202 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:25.185284 24202 hybrid_clock.cc:648] HybridClock initialized: now 1786261945185248 us; error 0 us; skew 500 ppm
I20260809 07:52:25.185696 24202 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:25.189337 24202 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:52:25.191280 24202 webserver.cc:533] Webserver started at http://127.26.37.61:43775/ using document root <none> and password file <none>
I20260809 07:52:25.192019 24202 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:25.192116 24202 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:25.192490 24202 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:25.195041 24202 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "681f7ead444f498c977f79f77af2031c"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-bw88"
I20260809 07:52:25.195796 24202 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "681f7ead444f498c977f79f77af2031c"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-bw88"
I20260809 07:52:25.201222 24202 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 07:52:25.204730 24219 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.206056 24202 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:25.206239 24202 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "681f7ead444f498c977f79f77af2031c"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-bw88"
I20260809 07:52:25.206498 24202 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:25.224045 24202 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:25.227749 24202 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:25.228045 24202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:25.249162 24202 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:34039
I20260809 07:52:25.249182 24298 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:34039 every 8 connection(s)
I20260809 07:52:25.250780 24202 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:52:25.254410 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24202
I20260809 07:52:25.254665 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:52:25.255223 24301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:25.261232 24301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:25.284483 24301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:25.301508 24301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:52:25.302695 24224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
W20260809 07:52:25.304512 24301 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:52:25.353830 24301 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 1
W20260809 07:52:25.355927 24301 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
I20260809 07:52:25.360106 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } attempt: 5
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:25.375713 24190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:52:25.377938 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:25.385165 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:39875
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:25.424182 24301 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 2
W20260809 07:52:25.426205 24301 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
I20260809 07:52:25.429150 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 1
W20260809 07:52:25.431074 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
I20260809 07:52:25.499356 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 2
W20260809 07:52:25.501643 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
I20260809 07:52:25.517462 24301 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 3
W20260809 07:52:25.519510 24301 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
I20260809 07:52:25.607940 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 3
W20260809 07:52:25.610325 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
I20260809 07:52:25.662777 24301 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 4
W20260809 07:52:25.665318 24301 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
W20260809 07:52:25.748389 24314 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:25.748723 24314 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:25.748809 24314 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:25.757115 24314 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:25.757226 24314 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:25.757284 24314 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:25.757326 24314 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:25.768102 24314 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:39875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:25.769762 24314 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:25.771091 24314 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:25.779758 24322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.779685 24320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.780746 24332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:25.780833 24314 server_base.cc:1061] running on GCE node
I20260809 07:52:25.781737 24314 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:25.781946 24314 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:25.782049 24314 hybrid_clock.cc:648] HybridClock initialized: now 1786261945782027 us; error 0 us; skew 500 ppm
I20260809 07:52:25.782402 24314 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:25.785979 24314 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:52:25.787586 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 4
I20260809 07:52:25.787981 24314 webserver.cc:533] Webserver started at http://127.26.37.60:36283/ using document root <none> and password file <none>
I20260809 07:52:25.788997 24314 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:25.789150 24314 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:25.789608 24314 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
W20260809 07:52:25.789691 24190 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111)
I20260809 07:52:25.792093 24314 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "f447064b8c254188b5b8a9e2ca007b78"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-bw88"
I20260809 07:52:25.792876 24314 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "f447064b8c254188b5b8a9e2ca007b78"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-bw88"
I20260809 07:52:25.799038 24314 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 07:52:25.803201 24340 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.804610 24314 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:52:25.804750 24314 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "f447064b8c254188b5b8a9e2ca007b78"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-bw88"
I20260809 07:52:25.804951 24314 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:25.829203 24314 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:25.833439 24314 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:25.833698 24314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:25.854913 24314 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:39875
I20260809 07:52:25.854946 24403 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:39875 every 8 connection(s)
I20260809 07:52:25.856567 24314 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:52:25.860962 24407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:25.865816 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24314
I20260809 07:52:25.866027 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:52:25.867007 24407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:25.891599 24407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:25.907642 24407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:25.927096 24407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Bootstrap starting.
I20260809 07:52:25.930327 24407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:25.931366 24407 log.cc:826] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:25.933949 24407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: No bootstrap required, opened a new log
I20260809 07:52:25.940245 24407 raft_consensus.cc:359] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:25.940521 24407 raft_consensus.cc:385] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:25.940621 24407 raft_consensus.cc:740] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f447064b8c254188b5b8a9e2ca007b78, State: Initialized, Role: FOLLOWER
I20260809 07:52:25.941185 24407 consensus_queue.cc:260] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:25.942076 24422 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:25.942373 24422 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:25.943336 24407 sys_catalog.cc:565] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:52:25.954252 24445 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:25.954383 24445 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:25.950973 24314 master_runner.cc:187] Error getting master registration for 127.26.37.62:33717: OK, instance_id { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" instance_seqno: 1786261944650701 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:52:25.956696 24314 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:52:25.964610 24301 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 5
Aug 09 07:52:25 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:25.982523 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: Bootstrap starting.
I20260809 07:52:25.985806 24301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:25.986891 24301 log.cc:826] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:25.989523 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: No bootstrap required, opened a new log
I20260809 07:52:25.995098 24422 raft_consensus.cc:493] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:25.995397 24422 raft_consensus.cc:515] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:25.995608 24301 raft_consensus.cc:359] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:25.995946 24301 raft_consensus.cc:385] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:25.996120 24301 raft_consensus.cc:740] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681f7ead444f498c977f79f77af2031c, State: Initialized, Role: FOLLOWER
I20260809 07:52:25.996795 24301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:25.997249 24422 leader_election.cc:290] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e05caec0c443b1bfbd167de9c9f8da (127.26.37.62:33717), 681f7ead444f498c977f79f77af2031c (127.26.37.61:34039)
I20260809 07:52:25.997807 24448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:25.998163 24448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:25.999049 24301 sys_catalog.cc:565] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:25.999235 24264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f447064b8c254188b5b8a9e2ca007b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681f7ead444f498c977f79f77af2031c" is_pre_election: true
I20260809 07:52:25.999747 24264 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f447064b8c254188b5b8a9e2ca007b78 in term 0.
I20260809 07:52:26.000391 24343 leader_election.cc:304] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 681f7ead444f498c977f79f77af2031c, f447064b8c254188b5b8a9e2ca007b78; no voters:
I20260809 07:52:26.000850 24422 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:52:26.001024 24422 raft_consensus.cc:493] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:26.001168 24422 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:26.004712 24422 raft_consensus.cc:515] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:26.007175 24264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f447064b8c254188b5b8a9e2ca007b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681f7ead444f498c977f79f77af2031c"
I20260809 07:52:26.007367 24264 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:26.009567 24422 leader_election.cc:290] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [CANDIDATE]: Term 1 election: Requested vote from peers 74e05caec0c443b1bfbd167de9c9f8da (127.26.37.62:33717), 681f7ead444f498c977f79f77af2031c (127.26.37.61:34039)
I20260809 07:52:26.011008 24264 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f447064b8c254188b5b8a9e2ca007b78 in term 1.
I20260809 07:52:26.011497 24343 leader_election.cc:304] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 681f7ead444f498c977f79f77af2031c, f447064b8c254188b5b8a9e2ca007b78; no voters:
I20260809 07:52:26.011863 24422 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:26.012635 24422 raft_consensus.cc:697] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 1 LEADER]: Becoming Leader. State: Replica: f447064b8c254188b5b8a9e2ca007b78, State: Running, Role: LEADER
I20260809 07:52:26.013231 24422 consensus_queue.cc:237] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:26.017295 24454 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f447064b8c254188b5b8a9e2ca007b78. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:26.017731 24454 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:26.031255 24456 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:52:26.033697 24456 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:52:26.049245 24264 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 1 FOLLOWER]: Refusing update from remote peer f447064b8c254188b5b8a9e2ca007b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:26.053988 24454 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:26.046109 24202 master_runner.cc:187] Error getting master registration for 127.26.37.62:33717: OK, instance_id { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" instance_seqno: 1786261944650701 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:52:26.058493 24202 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:52:26.058926 24466 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:26.059074 24466 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:26.066912 24190 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } attempt: 5
Aug 09 07:52:26 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:26.081581 24448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f447064b8c254188b5b8a9e2ca007b78. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:26.081822 24448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:26.084919 24459 mvcc.cc:204] Tried to move back new op lower bound from 7316528931005448192 to 7316528930875883520. Current Snapshot: MvccSnapshot[applied={T|T < 7316528931005448192}]
I20260809 07:52:26.096364 24190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap starting.
I20260809 07:52:26.098003 24422 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:26.098308 24422 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:26.099143 24456 catalog_manager.cc:1383] Generated new cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:26.099246 24456 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:52:26.102715 24468 mvcc.cc:204] Tried to move back new op lower bound from 7316528931005448192 to 7316528930875883520. Current Snapshot: MvccSnapshot[applied={T|T < 7316528931005448192}]
I20260809 07:52:26.106825 24190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:26.110028 24190 log.cc:826] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:26.115633 24448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:26.115844 24448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:26.121955 24190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: No bootstrap required, opened a new log
I20260809 07:52:26.123490 24456 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:52:26.127523 24456 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:52:26.129566 24190 raft_consensus.cc:359] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:26.129863 24190 raft_consensus.cc:385] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:26.129979 24190 raft_consensus.cc:740] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e05caec0c443b1bfbd167de9c9f8da, State: Initialized, Role: FOLLOWER
I20260809 07:52:26.130779 24190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:26.137868 24474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:26.138160 24474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:26.139977 24190 sys_catalog.cc:565] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:26.140413 24153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f447064b8c254188b5b8a9e2ca007b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e05caec0c443b1bfbd167de9c9f8da"
I20260809 07:52:26.141450 24154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f447064b8c254188b5b8a9e2ca007b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e05caec0c443b1bfbd167de9c9f8da" is_pre_election: true
I20260809 07:52:26.141897 24152 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:26.145731 24152 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Refusing update from remote peer f447064b8c254188b5b8a9e2ca007b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:26.146711 24454 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:26.191092 24456 catalog_manager.cc:6092] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Generated new TSK 0
I20260809 07:52:26.192238 24456 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:52:26.204572 24474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: New leader f447064b8c254188b5b8a9e2ca007b78. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:26.204600 24479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:26.204903 24474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:26.204919 24479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:26.204879 24422 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:26.205165 24422 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: LEADER
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
I20260809 07:52:26.275679 24493 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:26.275784 24493 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: loading cluster ID for follower catalog manager: success
I20260809 07:52:26.279481 24493 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: acquiring CA information for follower catalog manager: success
I20260809 07:52:26.281913 24493 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
W20260809 07:52:26.317883 26772 mini_kdc.cc:268] Time spent creating service keytab for kudu/127.26.37.1: real 0.110s user 0.000s sys 0.026s
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:26.395370 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:26.891419 24499 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:26.891757 24499 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:26.891938 24499 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:26.900528 24499 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:26.900770 24499 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:26.911664 24499 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:26.913472 24499 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:26.914976 24499 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:26.924731 24505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:26.924702 24504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:26.925541 24510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:26.926577 24499 server_base.cc:1061] running on GCE node
I20260809 07:52:26.927152 24499 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:26.927296 24499 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:26.927366 24499 hybrid_clock.cc:648] HybridClock initialized: now 1786261946927342 us; error 0 us; skew 500 ppm
I20260809 07:52:26.927624 24499 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:26 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261946, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:26.930972 24499 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:26.932482 24499 webserver.cc:533] Webserver started at http://127.26.37.1:41211/ using document root <none> and password file <none>
I20260809 07:52:26.933158 24499 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:26.933250 24499 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:26.933617 24499 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:26.935937 24499 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:26.936638 24499 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:26.942013 24499 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 07:52:26.945547 24516 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:26.946965 24499 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:26.947135 24499 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:26.947329 24499 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:26.968241 24499 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:26.971995 24499 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:26.972321 24499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:26.973260 24499 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:26.974931 24499 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:26.974999 24499 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:26.975145 24499 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:26.975207 24499 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:27.010843 24499 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:40149
I20260809 07:52:27.010895 24675 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:40149 every 8 connection(s)
I20260809 07:52:27.012439 24499 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:27.016575 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24499
I20260809 07:52:27.016819 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:52:27 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261946, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:27 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261946, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:27 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261946, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:27.050361 24678 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34039
I20260809 07:52:27.050860 24678 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:27.051898 24678 heartbeater.cc:507] Master 127.26.37.61:34039 requested a full tablet report, sending...
I20260809 07:52:27.054415 24676 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39875
I20260809 07:52:27.054561 24676 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:27.054862 24249 ts_manager.cc:194] Registered new tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:40149)
I20260809 07:52:27.055030 24676 heartbeater.cc:507] Master 127.26.37.60:39875 requested a full tablet report, sending...
I20260809 07:52:27.056813 24362 ts_manager.cc:194] Registered new tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:40149)
I20260809 07:52:27.058656 24362 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:39961
I20260809 07:52:27.061030 24466 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:27.061132 24466 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: loading cluster ID for follower catalog manager: success
I20260809 07:52:27.061546 24677 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:27.061753 24677 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:27.062163 24677 heartbeater.cc:507] Master 127.26.37.62:33717 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
I20260809 07:52:27.063964 24140 ts_manager.cc:194] Registered new tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:40149)
I20260809 07:52:27.064710 24466 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: acquiring CA information for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.2@KRBTEST.COM" created.
I20260809 07:52:27.066905 24466 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:27.164836 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:27.522401 24695 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:27.522735 24695 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:27.522931 24695 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:27.531443 24695 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:27.531689 24695 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:52:27.542608 24695 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:27.544373 24695 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:27.545838 24695 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:27.555281 24700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:27.556275 24701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:27.557394 24704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:27.557662 24695 server_base.cc:1061] running on GCE node
I20260809 07:52:27.558327 24695 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:27.558516 24695 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:27.558624 24695 hybrid_clock.cc:648] HybridClock initialized: now 1786261947558585 us; error 0 us; skew 500 ppm
I20260809 07:52:27.558930 24695 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:27 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:27.562331 24695 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:52:27.564009 24695 webserver.cc:533] Webserver started at http://127.26.37.2:39155/ using document root <none> and password file <none>
I20260809 07:52:27.564764 24695 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:27.564908 24695 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:27.565264 24695 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:27.567720 24695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "e194932a1564483883d17a5de2a84ab6"
format_stamp: "Formatted at 2026-08-09 07:52:27 on dist-test-slave-bw88"
I20260809 07:52:27.568545 24695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "e194932a1564483883d17a5de2a84ab6"
format_stamp: "Formatted at 2026-08-09 07:52:27 on dist-test-slave-bw88"
I20260809 07:52:27.573961 24695 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:52:27.577513 24711 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:27.578905 24695 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:27.579097 24695 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "e194932a1564483883d17a5de2a84ab6"
format_stamp: "Formatted at 2026-08-09 07:52:27 on dist-test-slave-bw88"
I20260809 07:52:27.579305 24695 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:27.602288 24695 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:27.606083 24695 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:27.606386 24695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:27.607359 24695 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:27.609233 24695 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:27.609341 24695 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:27.609499 24695 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:27.609637 24695 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:27.645908 24695 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:34007
I20260809 07:52:27.645957 24863 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:34007 every 8 connection(s)
I20260809 07:52:27.647509 24695 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:27.656198 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24695
I20260809 07:52:27.656505 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:52:27 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:27 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:27 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261947, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:27.692708 24866 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34039
I20260809 07:52:27.693216 24866 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:27.694284 24866 heartbeater.cc:507] Master 127.26.37.61:34039 requested a full tablet report, sending...
I20260809 07:52:27.695394 24865 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:27.695538 24865 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:27.696135 24865 heartbeater.cc:507] Master 127.26.37.62:33717 requested a full tablet report, sending...
I20260809 07:52:27.696424 24249 ts_manager.cc:194] Registered new tserver with Master: e194932a1564483883d17a5de2a84ab6 (127.26.37.2:34007)
I20260809 07:52:27.697506 24140 ts_manager.cc:194] Registered new tserver with Master: e194932a1564483883d17a5de2a84ab6 (127.26.37.2:34007)
I20260809 07:52:27.698971 24864 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39875
I20260809 07:52:27.699113 24864 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:27.699527 24864 heartbeater.cc:507] Master 127.26.37.60:39875 requested a full tablet report, sending...
I20260809 07:52:27.700707 24362 ts_manager.cc:194] Registered new tserver with Master: e194932a1564483883d17a5de2a84ab6 (127.26.37.2:34007)
I20260809 07:52:27.701716 24362 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:47271
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:27.800257 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:28.061940 24676 heartbeater.cc:499] Master 127.26.37.60:39875 was elected leader, sending a full tablet report...
W20260809 07:52:28.161232 24883 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:28.161700 24883 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:28.161870 24883 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:28.170491 24883 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:28.170739 24883 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:52:28.182001 24883 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:28.183709 24883 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:28.185106 24883 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:28.195791 24890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.195899 24891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.196779 24893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:28.197243 24883 server_base.cc:1061] running on GCE node
I20260809 07:52:28.197854 24883 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:28.198055 24883 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:28.198154 24883 hybrid_clock.cc:648] HybridClock initialized: now 1786261948198132 us; error 0 us; skew 500 ppm
I20260809 07:52:28.198562 24883 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:28 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:28.202129 24883 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:52:28.203761 24883 webserver.cc:533] Webserver started at http://127.26.37.3:33001/ using document root <none> and password file <none>
I20260809 07:52:28.204496 24883 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:28.204589 24883 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:28.204886 24883 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:28.207405 24883 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "7e8bfb9736204cefab0c446d253ec45f"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-bw88"
I20260809 07:52:28.208144 24883 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "7e8bfb9736204cefab0c446d253ec45f"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-bw88"
I20260809 07:52:28.214003 24883 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 07:52:28.217613 24902 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.219022 24883 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:28.219192 24883 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "7e8bfb9736204cefab0c446d253ec45f"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-bw88"
I20260809 07:52:28.219384 24883 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:28.236943 24883 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:28.243945 24883 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:28.244284 24883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:28.245319 24883 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:28.247586 24883 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:28.247675 24883 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.247830 24883 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:28.247874 24883 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.284366 24883 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:38753
I20260809 07:52:28.284456 25063 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:38753 every 8 connection(s)
I20260809 07:52:28.285904 24883 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:28.290047 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24883
I20260809 07:52:28.290277 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:52:28 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:28 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:28 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:28.315234 25065 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:28.315276 25068 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34039
I20260809 07:52:28.315706 25068 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:28.315711 25065 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:28.316363 25064 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39875
I20260809 07:52:28.316536 25064 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:28.317144 25064 heartbeater.cc:507] Master 127.26.37.60:39875 requested a full tablet report, sending...
I20260809 07:52:28.317186 25065 heartbeater.cc:507] Master 127.26.37.62:33717 requested a full tablet report, sending...
I20260809 07:52:28.318037 25068 heartbeater.cc:507] Master 127.26.37.61:34039 requested a full tablet report, sending...
I20260809 07:52:28.319177 24140 ts_manager.cc:194] Registered new tserver with Master: 7e8bfb9736204cefab0c446d253ec45f (127.26.37.3:38753)
I20260809 07:52:28.319356 24249 ts_manager.cc:194] Registered new tserver with Master: 7e8bfb9736204cefab0c446d253ec45f (127.26.37.3:38753)
I20260809 07:52:28.320207 24362 ts_manager.cc:194] Registered new tserver with Master: 7e8bfb9736204cefab0c446d253ec45f (127.26.37.3:38753)
I20260809 07:52:28.321041 24362 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:39799
I20260809 07:52:28.334511 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:52:28.341898 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:52:28.342115 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24095
I20260809 07:52:28.342968 24178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:28.453063 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24095
I20260809 07:52:28.481596 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33717
--webserver_interface=127.26.37.62
--webserver_port=38147
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:28.611824 24343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:33717: connect: Connection refused (error 111)
W20260809 07:52:28.613507 24343 consensus_peers.cc:597] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 -> Peer 74e05caec0c443b1bfbd167de9c9f8da (127.26.37.62:33717): Couldn't send request to peer 74e05caec0c443b1bfbd167de9c9f8da. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:33717: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:52:28.700172 24865 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:33717 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:33717: connect: Connection refused (error 111)
I20260809 07:52:28.703852 24864 heartbeater.cc:499] Master 127.26.37.60:39875 was elected leader, sending a full tablet report...
W20260809 07:52:28.844143 25079 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:28.844447 25079 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:28.844553 25079 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:28.852731 25079 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:28.852829 25079 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:28.852886 25079 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:28.852934 25079 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:28.863683 25079 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=38147
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:28.865283 25079 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:28.866596 25079 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:28.875264 25091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.875371 25090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.875916 25093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:28.876657 25079 server_base.cc:1061] running on GCE node
I20260809 07:52:28.877411 25079 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:28.877611 25079 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:28.877710 25079 hybrid_clock.cc:648] HybridClock initialized: now 1786261948877687 us; error 0 us; skew 500 ppm
I20260809 07:52:28.878085 25079 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:28 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:28.881764 25079 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:52:28.883625 25079 webserver.cc:533] Webserver started at http://127.26.37.62:38147/ using document root <none> and password file <none>
I20260809 07:52:28.884449 25079 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:28.884618 25079 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:28.890021 25079 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:28.892817 25103 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.894192 25079 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:28.894374 25079 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "74e05caec0c443b1bfbd167de9c9f8da"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-bw88"
I20260809 07:52:28.895102 25079 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:28.910280 25079 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:28.914273 25079 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:28.914552 25079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:28.935115 25079 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33717
I20260809 07:52:28.935200 25173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33717 every 8 connection(s)
I20260809 07:52:28.936769 25079 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:28.941745 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25079
I20260809 07:52:28.942087 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24499
I20260809 07:52:28.945096 25174 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:52:28.949146 25174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap starting.
Aug 09 07:52:28 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:52:28.962706 24671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:28.968816 25174 log.cc:826] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:28.977003 25174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:28.977608 25174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap complete.
I20260809 07:52:28.984447 25174 raft_consensus.cc:359] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:28.985167 25174 raft_consensus.cc:740] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e05caec0c443b1bfbd167de9c9f8da, State: Initialized, Role: FOLLOWER
I20260809 07:52:28.986029 25174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:28.986900 25182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:28.987191 25182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:28.987851 25174 sys_catalog.cc:565] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:28.992702 24454 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:28.993027 24454 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:29.001598 25201 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:52:29.019075 25182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: New leader f447064b8c254188b5b8a9e2ca007b78. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:29.019387 25182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 07:52:29 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:29 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:29 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:29.076736 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24499
W20260809 07:52:29.107553 24226 connection.cc:573] server connection from 127.26.37.1:59785 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:52:29.108497 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:40149
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--webserver_port=41211
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:29.108824 25205 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:29.071269 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:39831 (local address 127.26.37.62:33717)
0809 07:52:29.071420 (+ 151us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:29.071427 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:29.076411 (+ 4984us) server_negotiation.cc:408] Connection header received
0809 07:52:29.077024 (+ 613us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:29.077033 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:29.077148 (+ 115us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.077223 (+ 75us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:29.107629 (+ 30406us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:39831: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:52:29.323149 25064 heartbeater.cc:499] Master 127.26.37.60:39875 was elected leader, sending a full tablet report...
I20260809 07:52:29.335460 25124 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7e8bfb9736204cefab0c446d253ec45f" instance_seqno: 1786261948275539) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:57931; Asking this server to re-register.
I20260809 07:52:29.336280 25065 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:29.337918 25124 ts_manager.cc:194] Registered new tserver with Master: 7e8bfb9736204cefab0c446d253ec45f (127.26.37.3:38753)
W20260809 07:52:29.464054 25212 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:29.464461 25212 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:29.464627 25212 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:29.472945 25212 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:29.473189 25212 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:29.483613 25212 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:40149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41211
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:29.485392 25212 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:29.486894 25212 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:29.497380 25227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:29.497295 25224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:29.497366 25225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:29.498646 25212 server_base.cc:1061] running on GCE node
I20260809 07:52:29.499315 25212 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:29.499524 25212 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:29.499630 25212 hybrid_clock.cc:648] HybridClock initialized: now 1786261949499605 us; error 0 us; skew 500 ppm
I20260809 07:52:29.500083 25212 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:29 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261949, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:29.503644 25212 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:29.505293 25212 webserver.cc:533] Webserver started at http://127.26.37.1:41211/ using document root <none> and password file <none>
I20260809 07:52:29.506114 25212 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:29.506227 25212 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:29.512110 25212 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 07:52:29.515213 25234 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.516781 25212 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:52:29.516978 25212 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:29.517942 25212 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:29.531205 25212 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:29.535058 25212 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:29.535351 25212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:29.536294 25212 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:29.537948 25212 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:29.538043 25212 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.538192 25212 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:29.538342 25212 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 07:52:29.574254 25212 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:40149
I20260809 07:52:29.574349 25381 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:40149 every 8 connection(s)
I20260809 07:52:29.575809 25212 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:29.578522 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25212
Aug 09 07:52:29 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:52:29 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261949, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:29 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261949, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:29 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261949, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:29.642058 25403 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:29.642604 25403 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:29.651777 25382 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39875
I20260809 07:52:29.652297 25382 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:29.652736 25383 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:29.652886 25383 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:29.653733 25382 heartbeater.cc:507] Master 127.26.37.60:39875 requested a full tablet report, sending...
I20260809 07:52:29.654501 25383 heartbeater.cc:507] Master 127.26.37.62:33717 requested a full tablet report, sending...
I20260809 07:52:29.656068 24362 ts_manager.cc:194] Re-registered known tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:40149)
I20260809 07:52:29.657063 25124 ts_manager.cc:194] Registered new tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:40149)
I20260809 07:52:29.657265 24362 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:45381
I20260809 07:52:29.659320 25386 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34039
I20260809 07:52:29.659484 25386 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:29.659837 25386 heartbeater.cc:507] Master 127.26.37.61:34039 requested a full tablet report, sending...
I20260809 07:52:29.661295 24248 ts_manager.cc:194] Re-registered known tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:40149)
I20260809 07:52:29.734174 24865 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:29.735558 25124 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e194932a1564483883d17a5de2a84ab6" instance_seqno: 1786261947636933) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:58509; Asking this server to re-register.
I20260809 07:52:29.736217 24865 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:29.737603 25124 ts_manager.cc:194] Registered new tserver with Master: e194932a1564483883d17a5de2a84ab6 (127.26.37.2:34007)
I20260809 07:52:29.771493 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25212
W20260809 07:52:29.772702 25415 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:29.768829 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.26.37.1:40149)
0809 07:52:29.769619 (+ 790us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:29.769626 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:29.769656 (+ 30us) server_negotiation.cc:408] Connection header received
0809 07:52:29.769720 (+ 64us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:29.769727 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:29.769956 (+ 229us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.770130 (+ 174us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:29.771256 (+ 1126us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48066: BlockingRecv error: recv got EOF from 127.0.0.1:48066 (error 108)
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":426,"threads_started":1}
W20260809 07:52:29.774055 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25212: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:40149: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:52:29.774130 25415 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:29.772315 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.26.37.1:40149)
0809 07:52:29.772988 (+ 673us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:29.772996 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:29.773017 (+ 21us) server_negotiation.cc:408] Connection header received
0809 07:52:29.773083 (+ 66us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:29.773087 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:29.773169 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.773293 (+ 124us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:29.773918 (+ 625us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48074: BlockingRecv error: recv got EOF from 127.0.0.1:48074 (error 108)
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":682,"threads_started":1}
I20260809 07:52:29.774228 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25212
I20260809 07:52:29.806675 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:40149
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--webserver_port=41211
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:30.006546 25201 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:30.006691 25201 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: loading cluster ID for follower catalog manager: success
I20260809 07:52:30.010708 25201 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: acquiring CA information for follower catalog manager: success
I20260809 07:52:30.013250 25201 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:52:30.180482 25418 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:30.180869 25418 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:30.181031 25418 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:52:30.181124 25418 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:30.190339 25418 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:30.190599 25418 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:30.202358 25418 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:40149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41211
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:30.204174 25418 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:30.205703 25418 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:30.216409 25425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:30.216450 25426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:30.217388 25429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:30.217986 25418 server_base.cc:1061] running on GCE node
I20260809 07:52:30.218623 25418 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:30.218801 25418 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:30.218914 25418 hybrid_clock.cc:648] HybridClock initialized: now 1786261950218876 us; error 0 us; skew 500 ppm
I20260809 07:52:30.219230 25418 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:30 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261950, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:30.222764 25418 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:30.224619 25418 webserver.cc:533] Webserver started at http://127.26.37.1:41211/ using document root <none> and password file <none>
I20260809 07:52:30.225545 25418 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:30.225654 25418 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:30.231336 25418 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:30.234313 25437 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:30.235801 25418 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:30.235994 25418 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:30.236721 25418 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:30.256752 25418 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:30.261772 25418 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:30.262077 25418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:30.263167 25418 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:30.264994 25418 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:30.265093 25418 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:30.265223 25418 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:30.265283 25418 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:52:30.266108 25418 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:52:30.266203 25418 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:52:30.266289 25418 fault_injection.cc:59] injecting fault for kudu (pid 25418): FLAGS_fault_before_start (process will exit)
I20260809 07:52:30.290293 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:40149
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--webserver_port=41211
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:30.650252 25445 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:30.650632 25445 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:30.650831 25445 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:30.659572 25445 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:30.659768 25445 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:30.671155 25445 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:40149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41211
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:30.672873 25445 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:30.674258 25445 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:30.684877 25450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:30.684877 25451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:30.686208 25453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:30.686617 25445 server_base.cc:1061] running on GCE node
I20260809 07:52:30.687202 25445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:30.687336 25445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:30.687402 25445 hybrid_clock.cc:648] HybridClock initialized: now 1786261950687381 us; error 0 us; skew 500 ppm
I20260809 07:52:30.687654 25445 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:30 dist-test-slave-bw88 krb5kdc[23688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261950, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:30.691397 25445 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:30.693073 25445 webserver.cc:533] Webserver started at http://127.26.37.1:41211/ using document root <none> and password file <none>
I20260809 07:52:30.693873 25445 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:30.693984 25445 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:30.699489 25445 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:30.702642 25460 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:30.704319 25445 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:30.704536 25445 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:30.705488 25445 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:30.735576 25445 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:30.740451 25445 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:30.740765 25445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:30.741986 25445 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:30.743866 25445 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:30.744025 25445 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:30.744144 25445 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:30.744297 25445 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:30.782491 25445 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:40149
I20260809 07:52:30.782648 25605 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:40149 every 8 connection(s)
I20260809 07:52:30.784575 25445 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:30.794008 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25445
I20260809 07:52:30.794515 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25445
Aug 09 07:52:30 dist-test-slave-bw88 krb5kdc[23688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261950, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
W20260809 07:52:30.813712 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25445: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:40149: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:30.813932 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25445
W20260809 07:52:30.841056 25620 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:30.790628 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:59697 (local address 127.26.37.62:33717)
0809 07:52:30.804875 (+ 14247us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:30.804884 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:30.804909 (+ 25us) server_negotiation.cc:408] Connection header received
0809 07:52:30.804991 (+ 82us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:30.805000 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:30.805100 (+ 100us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.808238 (+ 3138us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:30.808532 (+ 294us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.811619 (+ 3087us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.840633 (+ 29014us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:59697: BlockingRecv error: recv got EOF from 127.26.37.1:59697 (error 108)
Metrics: {"server-negotiator.queue_time_us":14115,"thread_start_us":13986,"threads_started":1}
W20260809 07:52:30.841399 25623 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:30.791491 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:50107 (local address 127.26.37.61:34039)
0809 07:52:30.804229 (+ 12738us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:30.804238 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:30.804998 (+ 760us) server_negotiation.cc:408] Connection header received
0809 07:52:30.805157 (+ 159us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:30.805166 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:30.805257 (+ 91us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.805415 (+ 158us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:30.807067 (+ 1652us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.808577 (+ 1510us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.810827 (+ 2250us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.811424 (+ 597us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.840451 (+ 29027us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:50107: BlockingRecv error: recv got EOF from 127.26.37.1:50107 (error 108)
Metrics: {"server-negotiator.queue_time_us":12567,"thread_start_us":12730,"threads_started":1}
W20260809 07:52:30.841771 25611 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:30.788370 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:50647 (local address 127.26.37.60:39875)
0809 07:52:30.788854 (+ 484us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:30.788858 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:30.789986 (+ 1128us) server_negotiation.cc:408] Connection header received
0809 07:52:30.793500 (+ 3514us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:30.793512 (+ 12us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:30.793632 (+ 120us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.793781 (+ 149us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:30.796781 (+ 3000us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.798529 (+ 1748us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.801695 (+ 3166us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.802362 (+ 667us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.809263 (+ 6901us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0809 07:52:30.809325 (+ 62us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0809 07:52:30.809367 (+ 42us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
0809 07:52:30.809421 (+ 54us) server_negotiation.cc:923] calling sasl_server_start()
0809 07:52:30.821186 (+ 11765us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:30.840744 (+ 19558us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:50647: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":273,"threads_started":1}
I20260809 07:52:30.842079 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24695
W20260809 07:52:30.845002 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24695: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:34007: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:30.845145 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24695
I20260809 07:52:30.873317 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24883
W20260809 07:52:30.876201 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24883: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:38753: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:30.876392 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24883
I20260809 07:52:30.901225 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25079
W20260809 07:52:30.903590 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25079: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:33717: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:52:30.903658 25620 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:30.902287 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:60068 (local address 127.26.37.62:33717)
0809 07:52:30.902478 (+ 191us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:30.902486 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:30.902508 (+ 22us) server_negotiation.cc:408] Connection header received
0809 07:52:30.902591 (+ 83us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:30.902595 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:30.902661 (+ 66us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.902770 (+ 109us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:30.903480 (+ 710us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60068: BlockingRecv error: recv got EOF from 127.0.0.1:60068 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 07:52:30.903753 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25079
I20260809 07:52:30.931326 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24202
I20260809 07:52:30.932583 24290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:30.959085 24314 master_runner.cc:187] Error getting master registration for 127.26.37.62:33717: Network error: Client connection negotiation failed: client connection to 127.26.37.62:33717: connect: Connection refused (error 111),
I20260809 07:52:30.959326 24314 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:52:31.042153 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24202
I20260809 07:52:31.073493 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24314
I20260809 07:52:31.074676 24398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:31.192075 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24314
Shutting down hive metastore.
Aug 09 07:52:31 dist-test-slave-bw88 krb5kdc[25639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:31 dist-test-slave-bw88 krb5kdc[25639](info): set up 2 sockets
Aug 09 07:52:31 dist-test-slave-bw88 krb5kdc[25639](info): commencing operation
krb5kdc: starting...
W20260809 07:52:33.779332 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.165s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:52:33 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "hive/127.0.0.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:52:34: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:52:37 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261957, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:52:41 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261961, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:52:41 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261961, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:43.843861 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33717
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:44.199712 25907 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:44.200043 25907 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:44.200230 25907 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:44.208405 25907 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:44.208508 25907 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:44.208563 25907 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:44.208631 25907 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:44.219197 25907 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:44.220834 25907 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:44.222249 25907 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:44.231642 25918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:44.231905 25920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:44.232218 25917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:44.232517 25907 server_base.cc:1061] running on GCE node
I20260809 07:52:44.233305 25907 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:44.233567 25907 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:44.233662 25907 hybrid_clock.cc:648] HybridClock initialized: now 1786261964233624 us; error 0 us; skew 500 ppm
I20260809 07:52:44.233981 25907 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:44 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261964, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:44.237617 25907 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:52:44.239462 25907 webserver.cc:533] Webserver started at http://127.26.37.62:43819/ using document root <none> and password file <none>
I20260809 07:52:44.240206 25907 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:44.240355 25907 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:44.245816 25907 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:44.248749 25927 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:44.250123 25907 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:52:44.250314 25907 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "74e05caec0c443b1bfbd167de9c9f8da"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-bw88"
I20260809 07:52:44.251055 25907 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:44.280681 25907 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:44.284765 25907 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:44.285074 25907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:44.306821 25907 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33717
I20260809 07:52:44.306849 25995 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33717 every 8 connection(s)
I20260809 07:52:44.308496 25907 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:44.313320 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25907
I20260809 07:52:44.313544 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:52:44.316746 25996 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:52:44.321887 25996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap starting.
I20260809 07:52:44.340283 25996 log.cc:826] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:52:44.352231 25996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:44.352874 25996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap complete.
I20260809 07:52:44.360374 25996 raft_consensus.cc:359] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:44.361063 25996 raft_consensus.cc:740] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e05caec0c443b1bfbd167de9c9f8da, State: Initialized, Role: FOLLOWER
I20260809 07:52:44.361999 25996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:44.362912 26002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:44.363348 26002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:44.363979 25996 sys_catalog.cc:565] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:44.375521 26018 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:44.375720 26018 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: loading cluster ID for follower catalog manager: success
I20260809 07:52:44.380118 26018 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: acquiring CA information for follower catalog manager: success
I20260809 07:52:44.383358 26018 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:44.399701 25907 master_runner.cc:187] Error getting master registration for 127.26.37.61:34039: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34039: connect: Connection refused (error 111),
I20260809 07:52:44.405441 25907 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:44.444669 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:34039
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:44.807983 26028 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:44.808346 26028 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:44.808499 26028 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:44.816893 26028 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:44.816987 26028 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:44.817162 26028 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:44.817236 26028 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:44.827859 26028 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:34039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:44.829614 26028 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:44.831049 26028 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:44.839483 26034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:44.839483 26035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:44.840265 26038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:44.840827 26028 server_base.cc:1061] running on GCE node
I20260809 07:52:44.841655 26028 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:44.841848 26028 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:44.841946 26028 hybrid_clock.cc:648] HybridClock initialized: now 1786261964841921 us; error 0 us; skew 500 ppm
I20260809 07:52:44.842284 26028 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:44 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261964, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:44.845960 26028 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:52:44.847950 26028 webserver.cc:533] Webserver started at http://127.26.37.61:33019/ using document root <none> and password file <none>
I20260809 07:52:44.848668 26028 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:44.848780 26028 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:44.854101 26028 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:52:44.856913 26046 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:44.858248 26028 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:44.858439 26028 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "681f7ead444f498c977f79f77af2031c"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-bw88"
I20260809 07:52:44.859170 26028 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:44.879195 26028 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:44.883072 26028 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:44.883363 26028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:44.904772 26028 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:34039
I20260809 07:52:44.904795 26117 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:34039 every 8 connection(s)
I20260809 07:52:44.906548 26028 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:52:44.911859 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26028
I20260809 07:52:44.912081 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:52:44.914314 26118 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:52:44.917972 26118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: Bootstrap starting.
I20260809 07:52:44.935894 26118 log.cc:826] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:52:44.945510 26118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:44.946092 26118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: Bootstrap complete.
I20260809 07:52:44.953807 26118 raft_consensus.cc:359] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:44.954543 26118 raft_consensus.cc:740] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681f7ead444f498c977f79f77af2031c, State: Initialized, Role: FOLLOWER
I20260809 07:52:44.955349 26118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:44.956261 26125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:44.956576 26125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:44.957420 26118 sys_catalog.cc:565] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:44.968914 26139 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:44.969004 26139 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: loading cluster ID for follower catalog manager: success
I20260809 07:52:44.972904 26139 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: acquiring CA information for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:44.975543 26139 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:52:44 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261964, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:45.004191 26028 master_runner.cc:187] Error getting master registration for 127.26.37.60:39875: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39875: connect: Connection refused (error 111),
I20260809 07:52:45.008095 26028 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:45.049255 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:39875
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:45.427120 26149 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:45.427424 26149 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:45.427508 26149 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:45.435688 26149 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:45.435791 26149 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:45.435879 26149 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:45.435952 26149 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:45.446759 26149 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:39875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:45.448398 26149 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:45.449821 26149 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:45.458822 26155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:45.458793 26156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:45.459438 26159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:45.460070 26149 server_base.cc:1061] running on GCE node
I20260809 07:52:45.461011 26149 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:45.461217 26149 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:45.461297 26149 hybrid_clock.cc:648] HybridClock initialized: now 1786261965461274 us; error 0 us; skew 500 ppm
I20260809 07:52:45.461679 26149 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:45 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261965, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:45.465539 26149 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:52:45.467617 26149 webserver.cc:533] Webserver started at http://127.26.37.60:43037/ using document root <none> and password file <none>
I20260809 07:52:45.468426 26149 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:45.468585 26149 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:45.474058 26149 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:45.476867 26167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:45.478210 26149 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:45.478389 26149 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "f447064b8c254188b5b8a9e2ca007b78"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-bw88"
I20260809 07:52:45.479122 26149 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:45.498176 26149 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:45.501925 26149 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:45.502224 26149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:45.522876 26149 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:39875
I20260809 07:52:45.522953 26234 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:39875 every 8 connection(s)
I20260809 07:52:45.524549 26149 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:52:45.531555 26235 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:52:45.534008 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26149
I20260809 07:52:45.534210 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:52:45.535410 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Bootstrap starting.
Aug 09 07:52:45 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:45.554032 26235 log.cc:826] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:52:45 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:45.565333 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:45.566082 26235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Bootstrap complete.
I20260809 07:52:45.574975 26235 raft_consensus.cc:359] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:45.575750 26235 raft_consensus.cc:740] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f447064b8c254188b5b8a9e2ca007b78, State: Initialized, Role: FOLLOWER
I20260809 07:52:45.576505 26235 consensus_queue.cc:260] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:45.577432 26246 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:45.577847 26246 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:45.578608 26235 sys_catalog.cc:565] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 07:52:45 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:45.591343 26262 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:45.591521 26262 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: loading cluster ID for follower catalog manager: success
I20260809 07:52:45.595607 26262 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: acquiring CA information for follower catalog manager: success
Aug 09 07:52:45 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261965, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:45.598268 26262 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:52:45.625916 26149 master_runner.cc:264] No leader master found from master f447064b8c254188b5b8a9e2ca007b78
I20260809 07:52:45.626034 26149 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:52:45.820070 26265 raft_consensus.cc:493] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:45.820391 26265 raft_consensus.cc:515] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:45.822410 26265 leader_election.cc:290] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 681f7ead444f498c977f79f77af2031c (127.26.37.61:34039), f447064b8c254188b5b8a9e2ca007b78 (127.26.37.60:39875)
I20260809 07:52:45.831096 26080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74e05caec0c443b1bfbd167de9c9f8da" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "681f7ead444f498c977f79f77af2031c" is_pre_election: true
I20260809 07:52:45.831662 26080 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e05caec0c443b1bfbd167de9c9f8da in term 1.
I20260809 07:52:45.832347 25930 leader_election.cc:304] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 681f7ead444f498c977f79f77af2031c, 74e05caec0c443b1bfbd167de9c9f8da; no voters:
I20260809 07:52:45.832173 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74e05caec0c443b1bfbd167de9c9f8da" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f447064b8c254188b5b8a9e2ca007b78" is_pre_election: true
I20260809 07:52:45.832717 26265 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:52:45.832795 26202 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e05caec0c443b1bfbd167de9c9f8da in term 1.
I20260809 07:52:45.832880 26265 raft_consensus.cc:493] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:45.832988 26265 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:52:45.837882 26265 raft_consensus.cc:515] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:45.838613 26265 leader_election.cc:290] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [CANDIDATE]: Term 2 election: Requested vote from peers 681f7ead444f498c977f79f77af2031c (127.26.37.61:34039), f447064b8c254188b5b8a9e2ca007b78 (127.26.37.60:39875)
I20260809 07:52:45.838917 26080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74e05caec0c443b1bfbd167de9c9f8da" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "681f7ead444f498c977f79f77af2031c"
I20260809 07:52:45.839141 26080 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:52:45.839692 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74e05caec0c443b1bfbd167de9c9f8da" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f447064b8c254188b5b8a9e2ca007b78"
I20260809 07:52:45.839895 26202 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:52:45.843153 26080 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e05caec0c443b1bfbd167de9c9f8da in term 2.
I20260809 07:52:45.843652 25930 leader_election.cc:304] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 681f7ead444f498c977f79f77af2031c, 74e05caec0c443b1bfbd167de9c9f8da; no voters:
I20260809 07:52:45.844105 26265 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:52:45.844691 26265 raft_consensus.cc:697] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 2 LEADER]: Becoming Leader. State: Replica: 74e05caec0c443b1bfbd167de9c9f8da, State: Running, Role: LEADER
I20260809 07:52:45.845204 26265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:45.845914 26202 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e05caec0c443b1bfbd167de9c9f8da in term 2.
I20260809 07:52:45.847604 26268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e05caec0c443b1bfbd167de9c9f8da. Latest consensus state: current_term: 2 leader_uuid: "74e05caec0c443b1bfbd167de9c9f8da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:45.847885 26268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:45.848652 26270 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:52:45.850690 26270 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:52:45.851497 26270 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:45.851572 26270 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:52:45.852721 26270 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:52:45.853615 26270 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Loaded TSK: 0
I20260809 07:52:45.854447 26270 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:45.995894 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:46.256598 26202 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 2 FOLLOWER]: Refusing update from remote peer 74e05caec0c443b1bfbd167de9c9f8da: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:52:46.257665 26268 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [LEADER]: Connected to new peer: Peer: permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:52:46.261780 26283 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e05caec0c443b1bfbd167de9c9f8da. Latest consensus state: current_term: 2 leader_uuid: "74e05caec0c443b1bfbd167de9c9f8da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:46.262037 26283 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:46.263183 26265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "74e05caec0c443b1bfbd167de9c9f8da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:46.263414 26265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:46.264411 26080 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 2 FOLLOWER]: Refusing update from remote peer 74e05caec0c443b1bfbd167de9c9f8da: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:52:46.264336 26283 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "74e05caec0c443b1bfbd167de9c9f8da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:46.264552 26283 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:46.265318 26268 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [LEADER]: Connected to new peer: Peer: permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:52:46.273648 26293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "74e05caec0c443b1bfbd167de9c9f8da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:46.273963 26293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:46.274356 26289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e05caec0c443b1bfbd167de9c9f8da. Latest consensus state: current_term: 2 leader_uuid: "74e05caec0c443b1bfbd167de9c9f8da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:46.274717 26289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:52:46.354406 26281 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:46.354703 26281 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:46.354804 26281 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:46.363967 26281 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:46.364166 26281 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:46.374850 26281 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:46.376485 26281 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:46.377861 26281 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:46.387884 26300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:46.387920 26299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:46.388746 26303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:46.389477 26281 server_base.cc:1061] running on GCE node
I20260809 07:52:46.390013 26281 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:46.390162 26281 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:46.390228 26281 hybrid_clock.cc:648] HybridClock initialized: now 1786261966390211 us; error 0 us; skew 500 ppm
I20260809 07:52:46.390502 26281 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:46 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261966, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:46.394327 26281 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:46.396199 26281 webserver.cc:533] Webserver started at http://127.26.37.1:37425/ using document root <none> and password file <none>
I20260809 07:52:46.397034 26281 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:46.397178 26281 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:46.403294 26281 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:52:46.406349 26312 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:46.407900 26281 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:46.408195 26281 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:46.408922 26281 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:46.434367 26281 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:46.438467 26281 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:46.438766 26281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:46.439764 26281 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:46.441433 26281 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:46.441576 26281 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:46.441684 26281 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:46.441772 26281 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:46.480175 26281 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43391
I20260809 07:52:46.480270 26476 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43391 every 8 connection(s)
I20260809 07:52:46.482012 26281 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:46.486297 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26281
I20260809 07:52:46.486541 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:52:46 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261966, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:46 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261966, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:46 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261966, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:46.512766 26477 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39875
I20260809 07:52:46.513163 26477 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:46.513597 26484 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34039
I20260809 07:52:46.513757 26484 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:46.514377 26484 heartbeater.cc:507] Master 127.26.37.61:34039 requested a full tablet report, sending...
I20260809 07:52:46.514622 26477 heartbeater.cc:507] Master 127.26.37.60:39875 requested a full tablet report, sending...
I20260809 07:52:46.514863 26478 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:46.514986 26478 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:46.515501 26478 heartbeater.cc:507] Master 127.26.37.62:33717 requested a full tablet report, sending...
I20260809 07:52:46.517105 26068 ts_manager.cc:194] Registered new tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:43391)
I20260809 07:52:46.517155 25950 ts_manager.cc:194] Registered new tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:43391)
I20260809 07:52:46.517786 26189 ts_manager.cc:194] Registered new tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:43391)
I20260809 07:52:46.519086 25950 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:54625
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:46.624337 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:46.747870 26268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "74e05caec0c443b1bfbd167de9c9f8da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:46.748159 26268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: LEADER
W20260809 07:52:46.983150 26501 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:46.983613 26501 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:46.983783 26501 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:46.992473 26501 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:46.992731 26501 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:52:47.003551 26501 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:47.005268 26501 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:47.006773 26501 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:47.016783 26507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:47.017037 26508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:47.018249 26501 server_base.cc:1061] running on GCE node
W20260809 07:52:47.018384 26512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:47.018988 26501 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:47.019193 26501 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:47.019294 26501 hybrid_clock.cc:648] HybridClock initialized: now 1786261967019269 us; error 0 us; skew 500 ppm
I20260809 07:52:47.019631 26501 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:47 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:47.023514 26501 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:52:47.025254 26501 webserver.cc:533] Webserver started at http://127.26.37.2:35891/ using document root <none> and password file <none>
I20260809 07:52:47.026101 26501 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:47.026270 26501 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:47.031764 26501 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:52:47.034633 26523 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:47.036053 26501 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:47.036193 26501 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "e194932a1564483883d17a5de2a84ab6"
format_stamp: "Formatted at 2026-08-09 07:52:27 on dist-test-slave-bw88"
I20260809 07:52:47.036873 26501 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:47.064841 26501 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:47.068974 26501 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:47.069202 26501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:47.070238 26501 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:47.071772 26501 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:47.071858 26501 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:47.071940 26501 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:47.071993 26501 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:47.113236 26501 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:34305
I20260809 07:52:47.113327 26666 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:34305 every 8 connection(s)
I20260809 07:52:47.114975 26501 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:47.116438 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26501
I20260809 07:52:47.116685 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:52:47 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:47 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:47 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:47.154850 26667 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39875
I20260809 07:52:47.154951 26670 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:47.155283 26670 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:47.156404 26670 heartbeater.cc:507] Master 127.26.37.62:33717 requested a full tablet report, sending...
I20260809 07:52:47.156559 26671 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34039
I20260809 07:52:47.156718 26671 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:47.157250 26671 heartbeater.cc:507] Master 127.26.37.61:34039 requested a full tablet report, sending...
I20260809 07:52:47.155290 26667 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:47.157953 26667 heartbeater.cc:507] Master 127.26.37.60:39875 requested a full tablet report, sending...
I20260809 07:52:47.158344 25950 ts_manager.cc:194] Registered new tserver with Master: e194932a1564483883d17a5de2a84ab6 (127.26.37.2:34305)
I20260809 07:52:47.158619 26068 ts_manager.cc:194] Registered new tserver with Master: e194932a1564483883d17a5de2a84ab6 (127.26.37.2:34305)
I20260809 07:52:47.159080 26189 ts_manager.cc:194] Registered new tserver with Master: e194932a1564483883d17a5de2a84ab6 (127.26.37.2:34305)
I20260809 07:52:47.159596 25950 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:60667
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:52:47.252413 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:47.521590 26478 heartbeater.cc:499] Master 127.26.37.62:33717 was elected leader, sending a full tablet report...
W20260809 07:52:47.610627 26685 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:47.610924 26685 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:47.611021 26685 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:47.619352 26685 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:47.619530 26685 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:52:47.630439 26685 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:47.632085 26685 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:47.633486 26685 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:47.643622 26691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:47.643649 26690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:47.644065 26695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:47.644801 26685 server_base.cc:1061] running on GCE node
I20260809 07:52:47.645464 26685 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:47.645658 26685 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:47.645754 26685 hybrid_clock.cc:648] HybridClock initialized: now 1786261967645731 us; error 0 us; skew 500 ppm
I20260809 07:52:47.646121 26685 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:47 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:47.649794 26685 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:52:47.651440 26685 webserver.cc:533] Webserver started at http://127.26.37.3:45007/ using document root <none> and password file <none>
I20260809 07:52:47.652205 26685 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:47.652354 26685 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:47.658123 26685 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:52:47.661024 26701 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:47.662407 26685 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:47.662570 26685 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "7e8bfb9736204cefab0c446d253ec45f"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-bw88"
I20260809 07:52:47.663312 26685 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:47.704886 26685 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:47.708878 26685 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:47.709178 26685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:47.710204 26685 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:47.711854 26685 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:47.711964 26685 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:47.712103 26685 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:47.712184 26685 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:47.747294 26685 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:41873
I20260809 07:52:47.747378 26850 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:41873 every 8 connection(s)
I20260809 07:52:47.748817 26685 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:47.752466 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26685
I20260809 07:52:47.752672 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:52:47 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:52:47 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:52:47 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:52:47.778481 26854 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:47.778872 26854 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:47.779963 26854 heartbeater.cc:507] Master 127.26.37.62:33717 requested a full tablet report, sending...
I20260809 07:52:47.780885 26855 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34039
I20260809 07:52:47.781077 26855 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:47.781715 26855 heartbeater.cc:507] Master 127.26.37.61:34039 requested a full tablet report, sending...
I20260809 07:52:47.781939 25949 ts_manager.cc:194] Registered new tserver with Master: 7e8bfb9736204cefab0c446d253ec45f (127.26.37.3:41873)
I20260809 07:52:47.782744 26068 ts_manager.cc:194] Registered new tserver with Master: 7e8bfb9736204cefab0c446d253ec45f (127.26.37.3:41873)
I20260809 07:52:47.783179 25949 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:57443
I20260809 07:52:47.785995 26852 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39875
I20260809 07:52:47.786130 26852 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:47.786414 26852 heartbeater.cc:507] Master 127.26.37.60:39875 requested a full tablet report, sending...
I20260809 07:52:47.787437 26189 ts_manager.cc:194] Registered new tserver with Master: 7e8bfb9736204cefab0c446d253ec45f (127.26.37.3:41873)
I20260809 07:52:47.794875 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:52:47.802078 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:52:47.802225 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25907
I20260809 07:52:47.803036 25988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:47.915714 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25907
I20260809 07:52:47.947280 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33717
--webserver_interface=127.26.37.62
--webserver_port=43819
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:48.162004 26670 heartbeater.cc:499] Master 127.26.37.62:33717 was elected leader, sending a full tablet report...
W20260809 07:52:48.163837 26670 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:33717 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:33717: connect: Connection refused (error 111)
W20260809 07:52:48.306720 26864 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:48.307070 26864 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:48.307216 26864 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:48.315182 26864 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:48.315305 26864 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:48.315416 26864 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:48.315495 26864 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:48.325467 26864 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=43819
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:48.326967 26864 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:48.328255 26864 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:48.336385 26872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:48.336370 26871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:48.337504 26874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:48.337904 26864 server_base.cc:1061] running on GCE node
I20260809 07:52:48.338670 26864 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:48.338881 26864 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:48.338984 26864 hybrid_clock.cc:648] HybridClock initialized: now 1786261968338943 us; error 0 us; skew 500 ppm
I20260809 07:52:48.339295 26864 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:48 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261968, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:48.343180 26864 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:52:48.344933 26864 webserver.cc:533] Webserver started at http://127.26.37.62:43819/ using document root <none> and password file <none>
I20260809 07:52:48.345759 26864 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:48.345861 26864 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:48.351078 26864 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:52:48.353791 26880 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:48.355146 26864 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:48.355319 26864 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "74e05caec0c443b1bfbd167de9c9f8da"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-bw88"
I20260809 07:52:48.356010 26864 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:48.380132 26864 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:48.387159 26864 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:48.387430 26864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:48.408020 26864 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33717
I20260809 07:52:48.408044 26948 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33717 every 8 connection(s)
I20260809 07:52:48.409715 26864 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:48.416699 26949 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:52:48.417994 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26864
I20260809 07:52:48.418282 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26281
I20260809 07:52:48.420626 26949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap starting.
Aug 09 07:52:48 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:52:48.439414 26472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:48.440907 26949 log.cc:826] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:48.450973 26949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:48.451483 26949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: Bootstrap complete.
I20260809 07:52:48.458807 26949 raft_consensus.cc:359] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:48.459563 26949 raft_consensus.cc:740] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e05caec0c443b1bfbd167de9c9f8da, State: Initialized, Role: FOLLOWER
I20260809 07:52:48.460340 26949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:48.461380 26959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:48.461727 26959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:48.462677 26949 sys_catalog.cc:565] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:48.475013 26974 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:48.475229 26974 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: loading cluster ID for follower catalog manager: success
I20260809 07:52:48.480530 26974 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: acquiring CA information for follower catalog manager: success
I20260809 07:52:48.484086 26974 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:52:48 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261968, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:48.546005 26864 master_runner.cc:264] No leader master found from master 74e05caec0c443b1bfbd167de9c9f8da
I20260809 07:52:48.546166 26864 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:52:48.554025 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26281
W20260809 07:52:48.584332 26052 connection.cc:573] server connection from 127.26.37.1:45497 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:52:48.584990 26171 connection.cc:573] server connection from 127.26.37.1:42791 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:52:48.585520 26978 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:48.553328 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:51963 (local address 127.26.37.62:33717)
0809 07:52:48.553522 (+ 194us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:48.553531 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:48.553546 (+ 15us) server_negotiation.cc:408] Connection header received
0809 07:52:48.553602 (+ 56us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:48.553606 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:48.553678 (+ 72us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:48.553782 (+ 104us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0809 07:52:48.584400 (+ 30618us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:51963: BlockingRecv error: recv got EOF from 127.26.37.1:51963 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 07:52:48.586607 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43391
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--webserver_port=37425
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:48.785817 26854 heartbeater.cc:499] Master 127.26.37.62:33717 was elected leader, sending a full tablet report...
I20260809 07:52:48.795099 26900 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7e8bfb9736204cefab0c446d253ec45f" instance_seqno: 1786261967738639) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:51651; Asking this server to re-register.
I20260809 07:52:48.795925 26854 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:48.797453 26900 ts_manager.cc:194] Registered new tserver with Master: 7e8bfb9736204cefab0c446d253ec45f (127.26.37.3:41873)
W20260809 07:52:49.042466 26985 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:49.042781 26985 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:49.042938 26985 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:49.051518 26985 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:49.051760 26985 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:49.061858 26985 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=37425
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:49.063407 26985 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:49.065022 26985 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:49.074677 26993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:49.074683 26992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:49.075232 26999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:49.076105 26985 server_base.cc:1061] running on GCE node
I20260809 07:52:49.076647 26985 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:49.076843 26985 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:49.076925 26985 hybrid_clock.cc:648] HybridClock initialized: now 1786261969076892 us; error 0 us; skew 500 ppm
I20260809 07:52:49.077219 26985 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:49 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:49.080761 26985 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:49.082479 26985 webserver.cc:533] Webserver started at http://127.26.37.1:37425/ using document root <none> and password file <none>
I20260809 07:52:49.083206 26985 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:49.083353 26985 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:49.090603 26985 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 07:52:49.094641 27007 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:49.096648 26985 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:52:49.096882 26985 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:49.097929 26985 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:49.125627 27016 raft_consensus.cc:493] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74e05caec0c443b1bfbd167de9c9f8da)
I20260809 07:52:49.125854 27016 raft_consensus.cc:515] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:49.127470 27016 leader_election.cc:290] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74e05caec0c443b1bfbd167de9c9f8da (127.26.37.62:33717), 681f7ead444f498c977f79f77af2031c (127.26.37.61:34039)
I20260809 07:52:49.127583 26985 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:49.128822 26080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f447064b8c254188b5b8a9e2ca007b78" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "681f7ead444f498c977f79f77af2031c" is_pre_election: true
I20260809 07:52:49.132596 26985 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:49.132931 26985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:49.134341 26985 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:49.134824 26916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f447064b8c254188b5b8a9e2ca007b78" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74e05caec0c443b1bfbd167de9c9f8da" is_pre_election: true
I20260809 07:52:49.135329 26916 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f447064b8c254188b5b8a9e2ca007b78 in term 2.
I20260809 07:52:49.135998 26170 leader_election.cc:304] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e05caec0c443b1bfbd167de9c9f8da, f447064b8c254188b5b8a9e2ca007b78; no voters: 681f7ead444f498c977f79f77af2031c
I20260809 07:52:49.136439 26985 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:49.136435 27016 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:52:49.136566 26985 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:49.136577 27016 raft_consensus.cc:493] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74e05caec0c443b1bfbd167de9c9f8da)
I20260809 07:52:49.136708 26985 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:49.136677 27016 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:52:49.136795 26985 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:49.140146 27016 raft_consensus.cc:515] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:49.140980 27016 leader_election.cc:290] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [CANDIDATE]: Term 3 election: Requested vote from peers 74e05caec0c443b1bfbd167de9c9f8da (127.26.37.62:33717), 681f7ead444f498c977f79f77af2031c (127.26.37.61:34039)
I20260809 07:52:49.141453 26916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f447064b8c254188b5b8a9e2ca007b78" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74e05caec0c443b1bfbd167de9c9f8da"
I20260809 07:52:49.141664 26916 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:52:49.141764 26080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f447064b8c254188b5b8a9e2ca007b78" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "681f7ead444f498c977f79f77af2031c"
I20260809 07:52:49.145944 26916 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f447064b8c254188b5b8a9e2ca007b78 in term 3.
I20260809 07:52:49.146349 26170 leader_election.cc:304] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e05caec0c443b1bfbd167de9c9f8da, f447064b8c254188b5b8a9e2ca007b78; no voters: 681f7ead444f498c977f79f77af2031c
I20260809 07:52:49.146826 27016 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:52:49.147329 27016 raft_consensus.cc:697] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [term 3 LEADER]: Becoming Leader. State: Replica: f447064b8c254188b5b8a9e2ca007b78, State: Running, Role: LEADER
I20260809 07:52:49.147887 27016 consensus_queue.cc:237] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:49.150085 27032 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f447064b8c254188b5b8a9e2ca007b78. Latest consensus state: current_term: 3 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:49.150506 27032 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:49.151292 27052 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:52:49.153863 27052 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:52:49.154946 27052 catalog_manager.cc:1295] Loaded cluster ID: f5018e24072b44df8d083021c3b5a880
I20260809 07:52:49.155030 27052 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:52:49.156585 27052 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:52:49.157740 27052 catalog_manager.cc:6103] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78: Loaded TSK: 0
I20260809 07:52:49.158857 27052 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:52:49.173856 26670 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:49.174121 26670 heartbeater.cc:499] Master 127.26.37.62:33717 was elected leader, sending a full tablet report...
I20260809 07:52:49.175222 26900 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e194932a1564483883d17a5de2a84ab6" instance_seqno: 1786261967104421) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:47579; Asking this server to re-register.
I20260809 07:52:49.175859 26670 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:49.177119 26900 ts_manager.cc:194] Registered new tserver with Master: e194932a1564483883d17a5de2a84ab6 (127.26.37.2:34305)
I20260809 07:52:49.182754 26985 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43391
I20260809 07:52:49.182833 27171 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43391 every 8 connection(s)
I20260809 07:52:49.184384 26985 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:49.194093 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26985
Aug 09 07:52:49 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:49 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:52:49 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:52:49.214854 27173 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33717
I20260809 07:52:49.215356 27173 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:49.216486 27173 heartbeater.cc:507] Master 127.26.37.62:33717 requested a full tablet report, sending...
I20260809 07:52:49.218628 26900 ts_manager.cc:194] Registered new tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:43391)
Aug 09 07:52:49 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:52:49.224406 27172 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39875
I20260809 07:52:49.224545 27172 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:49.225116 27172 heartbeater.cc:507] Master 127.26.37.60:39875 requested a full tablet report, sending...
I20260809 07:52:49.226456 26189 ts_manager.cc:194] Re-registered known tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:43391)
I20260809 07:52:49.228384 26189 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:35963
I20260809 07:52:49.233332 27174 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34039
I20260809 07:52:49.233784 27174 heartbeater.cc:461] Registering TS with master...
I20260809 07:52:49.234172 27174 heartbeater.cc:507] Master 127.26.37.61:34039 requested a full tablet report, sending...
I20260809 07:52:49.235675 26068 ts_manager.cc:194] Re-registered known tserver with Master: bbc65267b0bb4eb4b6963805e89bc0dc (127.26.37.1:43391)
I20260809 07:52:49.237736 27185 raft_consensus.cc:493] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74e05caec0c443b1bfbd167de9c9f8da)
I20260809 07:52:49.237950 27185 raft_consensus.cc:515] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } }
I20260809 07:52:49.240429 27185 leader_election.cc:290] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74e05caec0c443b1bfbd167de9c9f8da (127.26.37.62:33717), f447064b8c254188b5b8a9e2ca007b78 (127.26.37.60:39875)
I20260809 07:52:49.247929 26916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "681f7ead444f498c977f79f77af2031c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74e05caec0c443b1bfbd167de9c9f8da" is_pre_election: true
I20260809 07:52:49.248294 26916 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 681f7ead444f498c977f79f77af2031c in current term 3: Already voted for candidate f447064b8c254188b5b8a9e2ca007b78 in this term.
I20260809 07:52:49.253961 26202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "681f7ead444f498c977f79f77af2031c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f447064b8c254188b5b8a9e2ca007b78" is_pre_election: true
I20260809 07:52:49.254823 26049 leader_election.cc:304] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 681f7ead444f498c977f79f77af2031c; no voters: 74e05caec0c443b1bfbd167de9c9f8da, f447064b8c254188b5b8a9e2ca007b78
I20260809 07:52:49.255357 27185 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:52:49.258678 27185 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:52:49.341065 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26985
W20260809 07:52:49.342697 27194 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:49.338613 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.26.37.1:43391)
0809 07:52:49.339449 (+ 836us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:49.339456 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:49.339488 (+ 32us) server_negotiation.cc:408] Connection header received
0809 07:52:49.339565 (+ 77us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:49.339572 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:49.339824 (+ 252us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:49.339996 (+ 172us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:49.340913 (+ 917us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54388: BlockingRecv error: recv got EOF from 127.0.0.1:54388 (error 108)
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":477,"threads_started":1}
W20260809 07:52:49.343736 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26985: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:43391: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:49.343883 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26985
I20260809 07:52:49.369316 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43391
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--webserver_port=37425
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:49.550603 26080 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [term 3 FOLLOWER]: Refusing update from remote peer f447064b8c254188b5b8a9e2ca007b78: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:52:49.551440 27032 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:52:49.553584 27185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f447064b8c254188b5b8a9e2ca007b78. Latest consensus state: current_term: 3 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:49.553938 27185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:49.555858 27016 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:49.556147 27016 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:49.556906 26916 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [term 3 FOLLOWER]: Refusing update from remote peer f447064b8c254188b5b8a9e2ca007b78: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:52:49.557722 27032 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:52:49.558290 27185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:49.558496 27185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681f7ead444f498c977f79f77af2031c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:49.561870 27205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:49.562176 27205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:52:49.562531 27201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: SysCatalogTable state changed. Reason: New leader f447064b8c254188b5b8a9e2ca007b78. Latest consensus state: current_term: 3 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:49.562768 27201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e05caec0c443b1bfbd167de9c9f8da [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:52:49.740937 27196 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:49.741259 27196 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:49.741421 27196 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:52:49.741503 27196 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:49.750550 27196 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:49.750759 27196 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:49.761642 27196 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=37425
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:49.763304 27196 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:49.764770 27196 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:49.775069 27212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:49.775163 27211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:49.775315 27214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:49.776275 27196 server_base.cc:1061] running on GCE node
I20260809 07:52:49.776912 27196 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:49.777086 27196 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:49.777204 27196 hybrid_clock.cc:648] HybridClock initialized: now 1786261969777150 us; error 0 us; skew 500 ppm
I20260809 07:52:49.777536 27196 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:49 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261969, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:49.781509 27196 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:49.783223 27196 webserver.cc:533] Webserver started at http://127.26.37.1:37425/ using document root <none> and password file <none>
I20260809 07:52:49.784096 27196 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:49.784224 27196 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:49.790028 27196 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:52:49.792922 27221 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:49.794307 27196 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:49.794512 27196 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:49.795202 27196 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:49.815037 27196 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:49.819104 27196 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:49.819401 27196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:49.820398 27196 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:49.822054 27196 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:49.822158 27196 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:49.822306 27196 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:49.822389 27196 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:52:49.822757 27196 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:52:49.822842 27196 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:52:49.822921 27196 fault_injection.cc:59] injecting fault for kudu (pid 27196): FLAGS_fault_before_start (process will exit)
I20260809 07:52:49.851159 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43391
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--webserver_port=37425
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:50.013401 27032 sys_catalog.cc:455] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f447064b8c254188b5b8a9e2ca007b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e05caec0c443b1bfbd167de9c9f8da" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33717 } } peers { permanent_uuid: "681f7ead444f498c977f79f77af2031c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34039 } } peers { permanent_uuid: "f447064b8c254188b5b8a9e2ca007b78" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39875 } } }
I20260809 07:52:50.013705 27032 sys_catalog.cc:458] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:50.167176 26667 heartbeater.cc:499] Master 127.26.37.60:39875 was elected leader, sending a full tablet report...
W20260809 07:52:50.210752 27231 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:50.211081 27231 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:50.211233 27231 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:50.219216 27231 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:50.219439 27231 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:52:50.229673 27231 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=37425
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33717,127.26.37.61:34039,127.26.37.60:39875
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:52:50.231364 27231 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:50.232811 27231 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:52:50.242834 27239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:50.243615 27237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:50.243749 27242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:50.244396 27231 server_base.cc:1061] running on GCE node
I20260809 07:52:50.244941 27231 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:52:50.245128 27231 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:52:50.245220 27231 hybrid_clock.cc:648] HybridClock initialized: now 1786261970245184 us; error 0 us; skew 500 ppm
I20260809 07:52:50.245560 27231 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:52:50 dist-test-slave-bw88 krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261970, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:50.249176 27231 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:52:50.250887 27231 webserver.cc:533] Webserver started at http://127.26.37.1:37425/ using document root <none> and password file <none>
I20260809 07:52:50.251622 27231 fs_manager.cc:362] Metadata directory not provided
I20260809 07:52:50.251772 27231 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:50.257246 27231 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:50.259950 27255 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:50.261526 27231 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:50.261698 27231 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "bbc65267b0bb4eb4b6963805e89bc0dc"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-bw88"
I20260809 07:52:50.262441 27231 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:50.285739 27231 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:52:50.289746 27231 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:50.290030 27231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:50.291051 27231 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:52:50.292631 27231 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:50.292719 27231 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:50.292858 27231 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:52:50.292920 27231 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:50.328171 27231 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43391
I20260809 07:52:50.328258 27410 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43391 every 8 connection(s)
I20260809 07:52:50.329692 27231 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:50.337132 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27231
I20260809 07:52:50.337527 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27231
Aug 09 07:52:50 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261970, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:52:50 dist-test-slave-bw88 krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261970, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
W20260809 07:52:50.354387 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27231: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:43391: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:50.354561 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27231
I20260809 07:52:50.380597 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26501
W20260809 07:52:50.381053 27422 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:50.335408 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:52683 (local address 127.26.37.62:33717)
0809 07:52:50.336054 (+ 646us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:50.336063 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:50.337235 (+ 1172us) server_negotiation.cc:408] Connection header received
0809 07:52:50.337554 (+ 319us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:50.337563 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:50.337682 (+ 119us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:50.337791 (+ 109us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:50.342684 (+ 4893us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.344490 (+ 1806us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:50.345898 (+ 1408us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.346437 (+ 539us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:52:50.349303 (+ 2866us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0809 07:52:50.349360 (+ 57us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0809 07:52:50.349368 (+ 8us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
0809 07:52:50.349415 (+ 47us) server_negotiation.cc:923] calling sasl_server_start()
0809 07:52:50.353685 (+ 4270us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:50.359384 (+ 5699us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
0809 07:52:50.359394 (+ 10us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
0809 07:52:50.359400 (+ 6us) server_negotiation.cc:968] calling sasl_server_step()
0809 07:52:50.359532 (+ 132us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:50.380495 (+ 20963us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:52683: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":352,"threads_started":1}
W20260809 07:52:50.381716 27418 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:50.333099 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:41023 (local address 127.26.37.60:39875)
0809 07:52:50.333616 (+ 517us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:50.333620 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:50.334588 (+ 968us) server_negotiation.cc:408] Connection header received
0809 07:52:50.336486 (+ 1898us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:50.336493 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:50.336589 (+ 96us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:50.336734 (+ 145us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:50.339105 (+ 2371us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.340743 (+ 1638us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:50.342714 (+ 1971us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.343314 (+ 600us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:52:50.347727 (+ 4413us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0809 07:52:50.347785 (+ 58us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0809 07:52:50.347793 (+ 8us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
0809 07:52:50.347840 (+ 47us) server_negotiation.cc:923] calling sasl_server_start()
0809 07:52:50.353267 (+ 5427us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:50.354407 (+ 1140us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
0809 07:52:50.354412 (+ 5us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
0809 07:52:50.354415 (+ 3us) server_negotiation.cc:968] calling sasl_server_step()
0809 07:52:50.354510 (+ 95us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:50.380555 (+ 26045us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:41023: BlockingRecv error: recv got EOF from 127.26.37.1:41023 (error 108)
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":269,"threads_started":1}
W20260809 07:52:50.382015 27423 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:50.336882 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:51335 (local address 127.26.37.61:34039)
0809 07:52:50.349693 (+ 12811us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:50.349701 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:50.349723 (+ 22us) server_negotiation.cc:408] Connection header received
0809 07:52:50.349785 (+ 62us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:50.349792 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:50.349865 (+ 73us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:50.349974 (+ 109us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:50.352443 (+ 2469us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.354231 (+ 1788us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:50.380311 (+ 26080us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:51335: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":12673,"thread_start_us":12505,"threads_started":1}
W20260809 07:52:50.383797 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26501: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:34305: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:50.383960 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26501
I20260809 07:52:50.406867 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26685
W20260809 07:52:50.409920 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26685: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:41873: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:52:50.410063 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26685
I20260809 07:52:50.438076 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26864
W20260809 07:52:50.440196 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26864: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:33717: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:52:50.440294 27422 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:52:50.438890 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49374 (local address 127.26.37.62:33717)
0809 07:52:50.439056 (+ 166us) server_negotiation.cc:207] Beginning negotiation
0809 07:52:50.439061 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:52:50.439149 (+ 88us) server_negotiation.cc:408] Connection header received
0809 07:52:50.439315 (+ 166us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:52:50.439319 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:52:50.439380 (+ 61us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:52:50.439454 (+ 74us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:52:50.440062 (+ 608us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49374: BlockingRecv error: recv got EOF from 127.0.0.1:49374 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:52:50.440373 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26864
I20260809 07:52:50.471271 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26028
I20260809 07:52:50.472517 26113 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 07:52:50.496829 26170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:33717: connect: Connection refused (error 111)
W20260809 07:52:50.498379 26170 consensus_peers.cc:597] T 00000000000000000000000000000000 P f447064b8c254188b5b8a9e2ca007b78 -> Peer 74e05caec0c443b1bfbd167de9c9f8da (127.26.37.62:33717): Couldn't send request to peer 74e05caec0c443b1bfbd167de9c9f8da. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:33717: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:52:50.582544 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26028
I20260809 07:52:50.612069 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26149
I20260809 07:52:50.613272 26230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:52:50.724746 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26149
Shutting down hive metastore.
[ OK ] ExternalMiniClusterTest.TestBasicOperation/9 (45645 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/10
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:51 dist-test-slave-bw88 krb5kdc[27448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:51 dist-test-slave-bw88 krb5kdc[27448](info): set up 2 sockets
Aug 09 07:52:51 dist-test-slave-bw88 krb5kdc[27448](info): commencing operation
krb5kdc: starting...
W20260809 07:52:53.332091 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.181s user 0.004s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:52:53 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:53Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:53Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:53:00: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:53:02 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261982, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:53:09 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261989, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:53:09 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261989, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:10.581225 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38157
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:10.930168 27827 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:10.930558 27827 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:10.930717 27827 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:10.938853 27827 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:10.938992 27827 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:10.939118 27827 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:10.939199 27827 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:10.954519 27827 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:10.956245 27827 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:10.957829 27827 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:10.966181 27834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:10.966393 27833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:10.966969 27839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:10.968132 27827 server_base.cc:1061] running on GCE node
I20260809 07:53:10.968874 27827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:10.970326 27827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:10.971534 27827 hybrid_clock.cc:648] HybridClock initialized: now 1786261990971469 us; error 56 us; skew 500 ppm
I20260809 07:53:10.971872 27827 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:10 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261990, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:10.975159 27827 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:53:10.976883 27827 webserver.cc:533] Webserver started at http://127.26.37.62:32861/ using document root <none> and password file <none>
I20260809 07:53:10.977660 27827 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:10.977783 27827 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:10.978160 27827 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:10.980495 27827 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "074f9dde34844437a15091fcae8153f3"
format_stamp: "Formatted at 2026-08-09 07:53:10 on dist-test-slave-bw88"
I20260809 07:53:10.981310 27827 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "074f9dde34844437a15091fcae8153f3"
format_stamp: "Formatted at 2026-08-09 07:53:10 on dist-test-slave-bw88"
I20260809 07:53:10.986691 27827 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 07:53:10.989945 27849 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:10.991161 27827 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:53:10.991345 27827 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "074f9dde34844437a15091fcae8153f3"
format_stamp: "Formatted at 2026-08-09 07:53:10 on dist-test-slave-bw88"
I20260809 07:53:10.991549 27827 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:11.007902 27827 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:11.015630 27827 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:11.015897 27827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:11.039016 27827 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:38157
I20260809 07:53:11.039073 27912 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:38157 every 8 connection(s)
I20260809 07:53:11.040596 27827 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:53:11.040891 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27827
I20260809 07:53:11.041132 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:53:11.045234 27913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:11.051139 27913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:11 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261990, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
I20260809 07:53:11.076376 27913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
W20260809 07:53:11.077903 27851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42563: connect: Connection refused (error 111)
W20260809 07:53:11.079602 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42563: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42563: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:11.128953 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } attempt: 1
W20260809 07:53:11.131120 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42563: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42563: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:11.176332 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:42563
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:11.199373 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } attempt: 2
W20260809 07:53:11.201619 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42563: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42563: connect: Connection refused (error 111)
I20260809 07:53:11.292917 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } attempt: 3
W20260809 07:53:11.295410 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42563: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42563: connect: Connection refused (error 111)
I20260809 07:53:11.438724 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } attempt: 4
W20260809 07:53:11.441032 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:42563: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42563: connect: Connection refused (error 111)
W20260809 07:53:11.522815 27925 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:11.523166 27925 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:11.523330 27925 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:11.531467 27925 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:11.531554 27925 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:11.531611 27925 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:11.531706 27925 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:11.542132 27925 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:42563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:11.543710 27925 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:11.545115 27925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:11.553643 27934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:11.553694 27935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:11.553820 27939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:11.555150 27925 server_base.cc:1061] running on GCE node
I20260809 07:53:11.555922 27925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:11.557260 27925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:11.558511 27925 hybrid_clock.cc:648] HybridClock initialized: now 1786261991558447 us; error 53 us; skew 500 ppm
I20260809 07:53:11.558858 27925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:11 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261991, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:11.562069 27925 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:53:11.563621 27925 webserver.cc:533] Webserver started at http://127.26.37.61:42959/ using document root <none> and password file <none>
I20260809 07:53:11.564371 27925 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:11.564550 27925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:11.564924 27925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:11.567400 27925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "98e67594146d4876987788c2d89e0b66"
format_stamp: "Formatted at 2026-08-09 07:53:11 on dist-test-slave-bw88"
I20260809 07:53:11.568192 27925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "98e67594146d4876987788c2d89e0b66"
format_stamp: "Formatted at 2026-08-09 07:53:11 on dist-test-slave-bw88"
I20260809 07:53:11.573479 27925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 07:53:11.576916 27948 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:11.578256 27925 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:53:11.578434 27925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "98e67594146d4876987788c2d89e0b66"
format_stamp: "Formatted at 2026-08-09 07:53:11 on dist-test-slave-bw88"
I20260809 07:53:11.578627 27925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:11.602051 27925 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:11.605784 27925 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:11.606073 27925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:11.626806 27925 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:42563
I20260809 07:53:11.626850 28010 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:42563 every 8 connection(s)
I20260809 07:53:11.628486 27925 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:53:11.632766 28011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:11.635823 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27925
I20260809 07:53:11.636026 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:53:11.638653 28011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:11 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261991, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:11.659730 28011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
Aug 09 07:53:11 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261991, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
I20260809 07:53:11.677304 28011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:53:11.678642 27952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
W20260809 07:53:11.680414 28011 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:53:11.729777 28011 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 1
W20260809 07:53:11.731886 28011 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
I20260809 07:53:11.740303 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } attempt: 5
Aug 09 07:53:11 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261990, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:11.754333 27913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:53:11.756498 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:11.765571 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:45019
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:11.800215 28011 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 2
W20260809 07:53:11.802245 28011 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
I20260809 07:53:11.807740 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 1
W20260809 07:53:11.809542 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
I20260809 07:53:11.877817 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 2
W20260809 07:53:11.880081 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
I20260809 07:53:11.893484 28011 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 3
W20260809 07:53:11.895560 28011 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
I20260809 07:53:11.986341 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 3
W20260809 07:53:11.988555 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
I20260809 07:53:12.038816 28011 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 4
W20260809 07:53:12.042354 28011 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
W20260809 07:53:12.122476 28025 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:12.122902 28025 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:12.123082 28025 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:12.130992 28025 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:12.131125 28025 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:12.131278 28025 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:12.131358 28025 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:12.141377 28025 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:45019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:12.142954 28025 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:12.144320 28025 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:12.152551 28031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:12.152715 28033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:12.153254 28035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:12.154444 28025 server_base.cc:1061] running on GCE node
I20260809 07:53:12.155258 28025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:12.156635 28025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:12.157860 28025 hybrid_clock.cc:648] HybridClock initialized: now 1786261992157830 us; error 49 us; skew 500 ppm
I20260809 07:53:12.158219 28025 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261992, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:12.161499 28025 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:53:12.163127 28025 webserver.cc:533] Webserver started at http://127.26.37.60:36269/ using document root <none> and password file <none>
I20260809 07:53:12.163851 28025 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:12.163995 28025 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:12.164387 28025 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:12.165786 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 4
I20260809 07:53:12.167171 28025 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "f693852090fa40b69fc88f7d1366c47c"
format_stamp: "Formatted at 2026-08-09 07:53:12 on dist-test-slave-bw88"
W20260809 07:53:12.167830 27913 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111)
I20260809 07:53:12.168188 28025 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "f693852090fa40b69fc88f7d1366c47c"
format_stamp: "Formatted at 2026-08-09 07:53:12 on dist-test-slave-bw88"
I20260809 07:53:12.173651 28025 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 07:53:12.177706 28043 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:12.179071 28025 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:53:12.179212 28025 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "f693852090fa40b69fc88f7d1366c47c"
format_stamp: "Formatted at 2026-08-09 07:53:12 on dist-test-slave-bw88"
I20260809 07:53:12.179368 28025 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:12.199491 28025 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:12.203367 28025 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:12.203610 28025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:12.224651 28025 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:45019
I20260809 07:53:12.224699 28109 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:45019 every 8 connection(s)
I20260809 07:53:12.226198 28025 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:53:12.230201 28110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:12.234521 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28025
I20260809 07:53:12.234726 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:53:12.236214 28110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261992, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:12.259757 28110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261992, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:12.274505 28110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261992, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:12.293859 28110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Bootstrap starting.
I20260809 07:53:12.297523 28110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:12.298610 28110 log.cc:826] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:12.301035 28110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: No bootstrap required, opened a new log
I20260809 07:53:12.306341 28110 raft_consensus.cc:359] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.306661 28110 raft_consensus.cc:385] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:53:12.306795 28110 raft_consensus.cc:740] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f693852090fa40b69fc88f7d1366c47c, State: Initialized, Role: FOLLOWER
I20260809 07:53:12.307365 28110 consensus_queue.cc:260] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.308182 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.308418 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:12.309211 28110 sys_catalog.cc:565] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:53:12.318470 28135 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:12.318614 28135 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:12.316409 28025 master_runner.cc:187] Error getting master registration for 127.26.37.62:38157: OK, instance_id { permanent_uuid: "074f9dde34844437a15091fcae8153f3" instance_seqno: 1786261991033214 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:53:12.323257 28025 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:12.341620 28011 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 5
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261991, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:12.361608 28011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: Bootstrap starting.
I20260809 07:53:12.364950 28011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:12.366142 28011 log.cc:826] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:12.368573 28011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: No bootstrap required, opened a new log
I20260809 07:53:12.374181 28011 raft_consensus.cc:359] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.374444 28011 raft_consensus.cc:385] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:53:12.374581 28011 raft_consensus.cc:740] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e67594146d4876987788c2d89e0b66, State: Initialized, Role: FOLLOWER
I20260809 07:53:12.375192 28011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.376124 28138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.376418 28138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:12.377202 28011 sys_catalog.cc:565] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:12.386817 27925 master_runner.cc:187] Error getting master registration for 127.26.37.62:38157: OK, instance_id { permanent_uuid: "074f9dde34844437a15091fcae8153f3" instance_seqno: 1786261991033214 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:53:12.391453 27925 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:53:12.393993 28151 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:12.394109 28151 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:12.414788 28124 raft_consensus.cc:493] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:53:12.415063 28124 raft_consensus.cc:515] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.416910 28124 leader_election.cc:290] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 074f9dde34844437a15091fcae8153f3 (127.26.37.62:38157), 98e67594146d4876987788c2d89e0b66 (127.26.37.61:42563)
I20260809 07:53:12.417961 27983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f693852090fa40b69fc88f7d1366c47c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e67594146d4876987788c2d89e0b66" is_pre_election: true
I20260809 07:53:12.418493 27983 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f693852090fa40b69fc88f7d1366c47c in term 0.
I20260809 07:53:12.419183 28045 leader_election.cc:304] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98e67594146d4876987788c2d89e0b66, f693852090fa40b69fc88f7d1366c47c; no voters:
I20260809 07:53:12.419595 28124 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:53:12.419771 28124 raft_consensus.cc:493] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:53:12.419883 28124 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:12.423465 28124 raft_consensus.cc:515] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.424388 28124 leader_election.cc:290] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [CANDIDATE]: Term 1 election: Requested vote from peers 074f9dde34844437a15091fcae8153f3 (127.26.37.62:38157), 98e67594146d4876987788c2d89e0b66 (127.26.37.61:42563)
I20260809 07:53:12.426164 27983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f693852090fa40b69fc88f7d1366c47c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e67594146d4876987788c2d89e0b66"
I20260809 07:53:12.426358 27983 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:12.430032 27983 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f693852090fa40b69fc88f7d1366c47c in term 1.
I20260809 07:53:12.430598 28045 leader_election.cc:304] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98e67594146d4876987788c2d89e0b66, f693852090fa40b69fc88f7d1366c47c; no voters:
I20260809 07:53:12.430980 28124 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:53:12.438853 28124 raft_consensus.cc:697] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 1 LEADER]: Becoming Leader. State: Replica: f693852090fa40b69fc88f7d1366c47c, State: Running, Role: LEADER
I20260809 07:53:12.439410 28124 consensus_queue.cc:237] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.445050 27913 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } attempt: 5
W20260809 07:53:12.455498 28050 tablet.cc:2406] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 07:53:12 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261990, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:12.473085 27913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap starting.
I20260809 07:53:12.481467 28153 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f693852090fa40b69fc88f7d1366c47c. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.482606 27913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:12.482824 28153 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:12.484498 28155 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:53:12.484974 27913 log.cc:826] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:12.488931 28155 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:53:12.489423 27913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: No bootstrap required, opened a new log
I20260809 07:53:12.500901 27913 raft_consensus.cc:359] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.501480 27913 raft_consensus.cc:385] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:53:12.501667 27913 raft_consensus.cc:740] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074f9dde34844437a15091fcae8153f3, State: Initialized, Role: FOLLOWER
I20260809 07:53:12.502492 27913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:12.511966 27913 sys_catalog.cc:565] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:12.512372 27983 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 1 FOLLOWER]: Refusing update from remote peer f693852090fa40b69fc88f7d1366c47c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:53:12.513090 27881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f693852090fa40b69fc88f7d1366c47c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074f9dde34844437a15091fcae8153f3" is_pre_election: true
I20260809 07:53:12.513422 27880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f693852090fa40b69fc88f7d1366c47c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074f9dde34844437a15091fcae8153f3"
I20260809 07:53:12.513804 27880 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:12.518077 28124 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:53:12.518738 27856 tablet.cc:2406] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:53:12.517311 27880 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f693852090fa40b69fc88f7d1366c47c in term 1.
I20260809 07:53:12.524806 27879 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Refusing update from remote peer f693852090fa40b69fc88f7d1366c47c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:53:12.525951 28160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.526296 28160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:12.527752 28124 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [LEADER]: Connected to new peer: Peer: permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:53:12.549995 28138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f693852090fa40b69fc88f7d1366c47c. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.550305 28138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:12.553673 28158 mvcc.cc:204] Tried to move back new op lower bound from 7316529121314885632 to 7316529121037361152. Current Snapshot: MvccSnapshot[applied={T|T < 7316529121314885632}]
I20260809 07:53:12.554095 28160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f693852090fa40b69fc88f7d1366c47c. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.554337 28160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:12.556859 28138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.557157 28138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:12.575263 28160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.575480 28160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:12.575803 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.579516 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:12.579759 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:12.579977 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:12.580152 28155 catalog_manager.cc:1383] Generated new cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:12.580255 28155 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:53:12.614658 28182 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:53:12.628727 28155 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:53:12.630373 28155 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:53:12.646695 28155 catalog_manager.cc:6092] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Generated new TSK 0
I20260809 07:53:12.647648 28155 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:12.780130 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:13.144372 28187 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:13.144749 28187 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:13.144935 28187 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:13.153496 28187 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:13.153664 28187 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:13.164144 28187 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:13.165804 28187 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:13.167078 28187 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:13.177485 28193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:13.177569 28192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:13.177809 28195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:13.178486 28187 server_base.cc:1061] running on GCE node
I20260809 07:53:13.179039 28187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:13.179972 28187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:13.181228 28187 hybrid_clock.cc:648] HybridClock initialized: now 1786261993181165 us; error 56 us; skew 500 ppm
I20260809 07:53:13.181603 28187 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:13 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:13.185096 28187 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:13.186826 28187 webserver.cc:533] Webserver started at http://127.26.37.1:38133/ using document root <none> and password file <none>
I20260809 07:53:13.187561 28187 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:13.187695 28187 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:13.188026 28187 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:13.190465 28187 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:13.191316 28187 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:13.196950 28187 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:53:13.200237 28207 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:13.201581 28187 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:53:13.201758 28187 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:13.201953 28187 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:13.220314 28187 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:13.223946 28187 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:13.224327 28187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:13.225268 28187 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:13.226876 28187 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:13.226976 28187 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:13.227114 28187 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:13.227180 28187 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:13.264832 28187 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:41681
I20260809 07:53:13.264917 28358 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:41681 every 8 connection(s)
I20260809 07:53:13.266358 28187 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:53:13.273849 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28187
I20260809 07:53:13.274077 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:53:13 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:13 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:53:13 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:13.298122 28359 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:13.298339 28360 heartbeater.cc:344] Connected to a master server at 127.26.37.60:45019
I20260809 07:53:13.298439 28362 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42563
I20260809 07:53:13.298568 28360 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:13.298599 28362 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:13.298574 28359 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:13.299652 28359 heartbeater.cc:507] Master 127.26.37.62:38157 requested a full tablet report, sending...
I20260809 07:53:13.299742 28362 heartbeater.cc:507] Master 127.26.37.61:42563 requested a full tablet report, sending...
I20260809 07:53:13.300370 28360 heartbeater.cc:507] Master 127.26.37.60:45019 requested a full tablet report, sending...
I20260809 07:53:13.302363 28065 ts_manager.cc:194] Registered new tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:41681)
I20260809 07:53:13.304669 28065 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:33469
I20260809 07:53:13.306558 27868 ts_manager.cc:194] Registered new tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:41681)
I20260809 07:53:13.308789 27968 ts_manager.cc:194] Registered new tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:41681)
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.2@KRBTEST.COM" created.
I20260809 07:53:13.396101 28151 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:13.396214 28151 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: loading cluster ID for follower catalog manager: success
I20260809 07:53:13.399298 28151 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: acquiring CA information for follower catalog manager: success
I20260809 07:53:13.401322 28151 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:13.404940 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:13.620798 28182 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:13.620973 28182 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: loading cluster ID for follower catalog manager: success
I20260809 07:53:13.625741 28182 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: acquiring CA information for follower catalog manager: success
I20260809 07:53:13.640311 28182 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:53:13.821571 28379 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:13.821991 28379 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:13.822149 28379 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:13.836038 28379 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:13.836294 28379 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:53:13.848039 28379 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:13.849695 28379 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:13.851154 28379 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:13.861022 28388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:13.861434 28389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:13.862692 28392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:13.863319 28379 server_base.cc:1061] running on GCE node
I20260809 07:53:13.863976 28379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:13.864917 28379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:13.866160 28379 hybrid_clock.cc:648] HybridClock initialized: now 1786261993866103 us; error 50 us; skew 500 ppm
I20260809 07:53:13.866521 28379 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:13 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:13.869973 28379 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:53:13.871809 28379 webserver.cc:533] Webserver started at http://127.26.37.2:45725/ using document root <none> and password file <none>
I20260809 07:53:13.872599 28379 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:13.872752 28379 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:13.873116 28379 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:13.875612 28379 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "6a8f8e269e6d40c0bc2dd545a0ecdcc8"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:13.876421 28379 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "6a8f8e269e6d40c0bc2dd545a0ecdcc8"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:13.882151 28379 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:53:13.885824 28406 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:13.887315 28379 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:53:13.887496 28379 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "6a8f8e269e6d40c0bc2dd545a0ecdcc8"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:13.887686 28379 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:13.906438 28379 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:13.910547 28379 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:13.910876 28379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:13.911949 28379 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:13.913761 28379 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:13.913836 28379 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:13.914006 28379 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:13.914062 28379 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:13.959590 28379 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:35733
I20260809 07:53:13.959630 28553 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:35733 every 8 connection(s)
I20260809 07:53:13.961064 28379 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:53:13.969302 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28379
I20260809 07:53:13.969640 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:53:13 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:53:13 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:14.012089 28556 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:14.012244 28559 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42563
I20260809 07:53:14.012547 28559 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:14.013540 28559 heartbeater.cc:507] Master 127.26.37.61:42563 requested a full tablet report, sending...
I20260809 07:53:14.012545 28556 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:14.015262 28556 heartbeater.cc:507] Master 127.26.37.62:38157 requested a full tablet report, sending...
I20260809 07:53:14.015336 27968 ts_manager.cc:194] Registered new tserver with Master: 6a8f8e269e6d40c0bc2dd545a0ecdcc8 (127.26.37.2:35733)
I20260809 07:53:14.016381 27868 ts_manager.cc:194] Registered new tserver with Master: 6a8f8e269e6d40c0bc2dd545a0ecdcc8 (127.26.37.2:35733)
Aug 09 07:53:14 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261993, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.3@KRBTEST.COM" created.
I20260809 07:53:14.028602 28558 heartbeater.cc:344] Connected to a master server at 127.26.37.60:45019
I20260809 07:53:14.028803 28558 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:14.029395 28558 heartbeater.cc:507] Master 127.26.37.60:45019 requested a full tablet report, sending...
I20260809 07:53:14.030597 28065 ts_manager.cc:194] Registered new tserver with Master: 6a8f8e269e6d40c0bc2dd545a0ecdcc8 (127.26.37.2:35733)
I20260809 07:53:14.031733 28065 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:44883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:14.155285 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:14.307996 28360 heartbeater.cc:499] Master 127.26.37.60:45019 was elected leader, sending a full tablet report...
W20260809 07:53:14.512104 28572 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:14.512451 28572 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:14.512575 28572 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:14.520669 28572 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:14.520855 28572 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:53:14.531189 28572 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:14.532742 28572 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:14.534178 28572 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:14.543881 28579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:14.543848 28578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:14.544503 28581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:14.545218 28572 server_base.cc:1061] running on GCE node
I20260809 07:53:14.545884 28572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:14.546877 28572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:14.548128 28572 hybrid_clock.cc:648] HybridClock initialized: now 1786261994548068 us; error 51 us; skew 500 ppm
I20260809 07:53:14.548458 28572 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:14 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261994, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:14.551910 28572 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:53:14.553580 28572 webserver.cc:533] Webserver started at http://127.26.37.3:46747/ using document root <none> and password file <none>
I20260809 07:53:14.554344 28572 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:14.554493 28572 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:14.554868 28572 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:14.557523 28572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "a9d033442a3f4232b275f79e9f1a1dde"
format_stamp: "Formatted at 2026-08-09 07:53:14 on dist-test-slave-bw88"
I20260809 07:53:14.558344 28572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "a9d033442a3f4232b275f79e9f1a1dde"
format_stamp: "Formatted at 2026-08-09 07:53:14 on dist-test-slave-bw88"
I20260809 07:53:14.563771 28572 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:53:14.567375 28590 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:14.568739 28572 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:53:14.568925 28572 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "a9d033442a3f4232b275f79e9f1a1dde"
format_stamp: "Formatted at 2026-08-09 07:53:14 on dist-test-slave-bw88"
I20260809 07:53:14.569128 28572 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:14.591096 28572 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:14.595150 28572 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:14.595446 28572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:14.596560 28572 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:14.598268 28572 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:14.598373 28572 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:14.598546 28572 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:14.598618 28572 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:14.636670 28572 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:42085
I20260809 07:53:14.636752 28742 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:42085 every 8 connection(s)
I20260809 07:53:14.638572 28572 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:53:14.647356 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28572
I20260809 07:53:14.647624 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:53:14 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261994, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:53:14 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261994, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:14 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261994, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:14.688936 28746 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42563
I20260809 07:53:14.689388 28746 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:14.690230 28745 heartbeater.cc:344] Connected to a master server at 127.26.37.60:45019
I20260809 07:53:14.690387 28745 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:14.690444 28746 heartbeater.cc:507] Master 127.26.37.61:42563 requested a full tablet report, sending...
I20260809 07:53:14.690855 28745 heartbeater.cc:507] Master 127.26.37.60:45019 requested a full tablet report, sending...
I20260809 07:53:14.692238 28744 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:14.692356 28065 ts_manager.cc:194] Registered new tserver with Master: a9d033442a3f4232b275f79e9f1a1dde (127.26.37.3:42085)
I20260809 07:53:14.692411 28744 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:14.692898 28744 heartbeater.cc:507] Master 127.26.37.62:38157 requested a full tablet report, sending...
I20260809 07:53:14.693454 28065 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:59523
I20260809 07:53:14.694001 27868 ts_manager.cc:194] Registered new tserver with Master: a9d033442a3f4232b275f79e9f1a1dde (127.26.37.3:42085)
I20260809 07:53:14.694991 27968 ts_manager.cc:194] Registered new tserver with Master: a9d033442a3f4232b275f79e9f1a1dde (127.26.37.3:42085)
I20260809 07:53:14.708747 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:53:14.715839 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:53:14.716048 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27827
I20260809 07:53:14.716882 27907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:53:14.825888 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27827
W20260809 07:53:14.857571 28044 connection.cc:573] client connection to 127.26.37.62:38157 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:53:14.857861 28044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:53:14.858601 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38157
--webserver_interface=127.26.37.62
--webserver_port=32861
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:14.860357 28044 consensus_peers.cc:597] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c -> Peer 074f9dde34844437a15091fcae8153f3 (127.26.37.62:38157): Couldn't send request to peer 074f9dde34844437a15091fcae8153f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:38157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:53:15.018869 28556 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:38157 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:38157: connect: Connection refused (error 111)
I20260809 07:53:15.034813 28558 heartbeater.cc:499] Master 127.26.37.60:45019 was elected leader, sending a full tablet report...
W20260809 07:53:15.220546 28763 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:15.220878 28763 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:15.221019 28763 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:15.232861 28763 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:15.233008 28763 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:15.233095 28763 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:15.233157 28763 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:15.244971 28763 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=32861
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:15.246583 28763 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:15.248011 28763 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:15.256923 28773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:15.257005 28772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:15.257503 28775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:15.258059 28763 server_base.cc:1061] running on GCE node
I20260809 07:53:15.258912 28763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:15.260386 28763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:15.261607 28763 hybrid_clock.cc:648] HybridClock initialized: now 1786261995261558 us; error 37 us; skew 500 ppm
I20260809 07:53:15.261991 28763 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:15 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261995, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:15.265861 28763 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:53:15.267884 28763 webserver.cc:533] Webserver started at http://127.26.37.62:32861/ using document root <none> and password file <none>
I20260809 07:53:15.268747 28763 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:15.268936 28763 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:15.274837 28763 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:53:15.277786 28783 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:15.279268 28763 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:53:15.279457 28763 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "074f9dde34844437a15091fcae8153f3"
format_stamp: "Formatted at 2026-08-09 07:53:10 on dist-test-slave-bw88"
I20260809 07:53:15.280159 28763 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:15.295965 28153 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:15.296205 28153 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:15.307732 28763 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
W20260809 07:53:15.311990 28359 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:38157 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:38157: connect: Connection refused (error 111)
I20260809 07:53:15.312456 28763 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:15.312868 28763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:15.334282 28763 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:38157
I20260809 07:53:15.334334 28853 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:38157 every 8 connection(s)
I20260809 07:53:15.336143 28763 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:53:15.340289 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28763
I20260809 07:53:15.340632 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28187
I20260809 07:53:15.344450 28856 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:53:15.348549 28856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap starting.
Aug 09 07:53:15 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:53:15.360049 28352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:53:15.369701 28856 log.cc:826] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:15.381330 28856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:53:15.382002 28856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap complete.
I20260809 07:53:15.392374 28856 raft_consensus.cc:359] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:15.393381 28856 raft_consensus.cc:740] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074f9dde34844437a15091fcae8153f3, State: Initialized, Role: FOLLOWER
I20260809 07:53:15.394239 28856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:15.395124 28863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:15.395386 28863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:15.396482 28856 sys_catalog.cc:565] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:15.413280 28877 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:15.415378 28877 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: loading cluster ID for follower catalog manager: success
I20260809 07:53:15.419592 28877 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: acquiring CA information for follower catalog manager: success
I20260809 07:53:15.422290 28877 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:53:15 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261995, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:15.480772 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28187
I20260809 07:53:15.509685 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:41681
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--webserver_port=38133
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:15.696182 28745 heartbeater.cc:499] Master 127.26.37.60:45019 was elected leader, sending a full tablet report...
I20260809 07:53:15.709827 28807 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a9d033442a3f4232b275f79e9f1a1dde" instance_seqno: 1786261994626958) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:41143; Asking this server to re-register.
I20260809 07:53:15.710726 28744 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:15.712605 28807 ts_manager.cc:194] Registered new tserver with Master: a9d033442a3f4232b275f79e9f1a1dde (127.26.37.3:42085)
W20260809 07:53:15.877245 28888 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:15.877646 28888 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:15.877797 28888 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:53:15.882577 28863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f693852090fa40b69fc88f7d1366c47c. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:15.882884 28863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:53:15.886687 28888 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:15.886943 28888 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:15.897401 28888 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:41681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38133
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:15.898990 28888 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:15.900360 28888 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:15.910493 28897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:15.910526 28898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:15.911407 28901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:15.911942 28888 server_base.cc:1061] running on GCE node
I20260809 07:53:15.912555 28888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:15.913524 28888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:15.914739 28888 hybrid_clock.cc:648] HybridClock initialized: now 1786261995914686 us; error 29 us; skew 500 ppm
I20260809 07:53:15.915061 28888 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:15 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261995, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:15.918526 28888 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:15.920205 28888 webserver.cc:533] Webserver started at http://127.26.37.1:38133/ using document root <none> and password file <none>
I20260809 07:53:15.920871 28888 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:15.920977 28888 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:15.926473 28888 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:53:15.929082 28908 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:15.930449 28888 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:15.930620 28888 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:15.931391 28888 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:15.951673 28888 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:15.955672 28888 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:15.955899 28888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:15.956801 28888 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:15.958550 28888 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:15.958681 28888 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:15.958799 28888 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:15.958956 28888 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 07:53:15.993865 28888 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:41681
I20260809 07:53:15.994020 29055 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:41681 every 8 connection(s)
I20260809 07:53:15.995471 28888 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:53:16.001010 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28888
Aug 09 07:53:16 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:53:16 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261995, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:53:16 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261995, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:16 dist-test-slave-bw88 krb5kdc[27448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261995, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:16.033128 28556 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:16.034456 28807 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6a8f8e269e6d40c0bc2dd545a0ecdcc8" instance_seqno: 1786261993949810) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:50307; Asking this server to re-register.
I20260809 07:53:16.035060 28556 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:16.036311 28807 ts_manager.cc:194] Registered new tserver with Master: 6a8f8e269e6d40c0bc2dd545a0ecdcc8 (127.26.37.2:35733)
I20260809 07:53:16.039940 29059 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42563
I20260809 07:53:16.040355 29059 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:16.041006 29056 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:16.041174 29056 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:16.041565 29059 heartbeater.cc:507] Master 127.26.37.61:42563 requested a full tablet report, sending...
I20260809 07:53:16.041846 29056 heartbeater.cc:507] Master 127.26.37.62:38157 requested a full tablet report, sending...
I20260809 07:53:16.044014 27968 ts_manager.cc:194] Re-registered known tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:41681)
I20260809 07:53:16.044749 28807 ts_manager.cc:194] Registered new tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:41681)
I20260809 07:53:16.049543 29058 heartbeater.cc:344] Connected to a master server at 127.26.37.60:45019
I20260809 07:53:16.049693 29058 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:16.050227 29058 heartbeater.cc:507] Master 127.26.37.60:45019 requested a full tablet report, sending...
I20260809 07:53:16.051348 28065 ts_manager.cc:194] Re-registered known tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:41681)
I20260809 07:53:16.052204 28065 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:50013
I20260809 07:53:16.163790 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28888
W20260809 07:53:16.165032 29075 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:16.161117 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.26.37.1:41681)
0809 07:53:16.161875 (+ 758us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:16.161885 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:16.161923 (+ 38us) server_negotiation.cc:408] Connection header received
0809 07:53:16.162155 (+ 232us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:16.162163 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:16.162403 (+ 240us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:16.162626 (+ 223us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:16.163673 (+ 1047us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59260: BlockingRecv error: recv got EOF from 127.0.0.1:59260 (error 108)
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":384,"threads_started":1}
W20260809 07:53:16.166110 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28888: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:41681: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:16.166263 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28888
I20260809 07:53:16.191857 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:41681
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--webserver_port=38133
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:16.274789 28153 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:16.275107 28153 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: LEADER
W20260809 07:53:16.556481 29077 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:16.556841 29077 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:16.557016 29077 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:53:16.557122 29077 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:16.565594 29077 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:16.565851 29077 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:16.576507 29077 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:41681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38133
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:16.578210 29077 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:16.579648 29077 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:16.590296 29087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:16.590307 29086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:16.590553 29089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:16.591912 29077 server_base.cc:1061] running on GCE node
I20260809 07:53:16.592483 29077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:16.593605 29077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:16.594853 29077 hybrid_clock.cc:648] HybridClock initialized: now 1786261996594795 us; error 38 us; skew 500 ppm
I20260809 07:53:16.595166 29077 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:16 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261996, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:16.598851 29077 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:16.600577 29077 webserver.cc:533] Webserver started at http://127.26.37.1:38133/ using document root <none> and password file <none>
I20260809 07:53:16.601439 29077 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:16.601557 29077 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:16.606964 29077 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:53:16.609807 29100 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:16.611267 29077 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:16.611465 29077 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:16.612210 29077 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:16.649744 29077 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:16.654111 29077 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:16.654376 29077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:16.655364 29077 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:16.657032 29077 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:16.657107 29077 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:16.657212 29077 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:16.657289 29077 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:53:16.657793 29077 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:53:16.657862 29077 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:53:16.657958 29077 fault_injection.cc:59] injecting fault for kudu (pid 29077): FLAGS_fault_before_start (process will exit)
I20260809 07:53:16.684413 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:41681
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--webserver_port=38133
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:17.040645 29113 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:17.041020 29113 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:17.041205 29113 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:17.049772 29113 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:17.050016 29113 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:17.062039 29113 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:41681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38133
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:17.063762 29113 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:17.065141 29113 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:17.075666 29119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:17.075803 29122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:17.075839 29120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:17.076526 29113 server_base.cc:1061] running on GCE node
I20260809 07:53:17.077034 29113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:17.077934 29113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:17.079138 29113 hybrid_clock.cc:648] HybridClock initialized: now 1786261997079074 us; error 53 us; skew 500 ppm
I20260809 07:53:17.079452 29113 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:17 dist-test-slave-bw88 krb5kdc[27448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261997, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:17.083098 29113 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:17.084945 29113 webserver.cc:533] Webserver started at http://127.26.37.1:38133/ using document root <none> and password file <none>
I20260809 07:53:17.086093 29113 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:17.086241 29113 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:17.094472 29113 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 07:53:17.097589 29136 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:17.099074 29113 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:17.099270 29113 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:17.100024 29113 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:17.114441 29113 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:17.118240 29113 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:17.118527 29113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:17.119473 29113 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:17.121068 29113 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:17.121206 29113 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:17.121376 29113 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:17.121472 29113 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 07:53:17.157686 29113 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:41681
I20260809 07:53:17.157771 29282 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:41681 every 8 connection(s)
I20260809 07:53:17.159293 29113 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:53:17.167716 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29113
I20260809 07:53:17.168160 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29113
W20260809 07:53:17.194497 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29113: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:41681: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:17.194700 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29113
I20260809 07:53:17.221581 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28379
W20260809 07:53:17.222324 29291 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:17.164876 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:37283 (local address 127.26.37.62:38157)
0809 07:53:17.209769 (+ 44893us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:17.209779 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:17.209797 (+ 18us) server_negotiation.cc:408] Connection header received
0809 07:53:17.209850 (+ 53us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:17.209855 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:17.209913 (+ 58us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:17.210007 (+ 94us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:17.221260 (+ 11253us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:37283: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":44782,"thread_start_us":44675,"threads_started":1}
W20260809 07:53:17.222316 29290 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:17.164835 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:32933 (local address 127.26.37.61:42563)
0809 07:53:17.202279 (+ 37444us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:17.202289 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:17.202318 (+ 29us) server_negotiation.cc:408] Connection header received
0809 07:53:17.202403 (+ 85us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:17.202414 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:17.202533 (+ 119us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:17.202669 (+ 136us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:17.221260 (+ 18591us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:32933: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":37252,"thread_start_us":37940,"threads_started":1}
W20260809 07:53:17.222887 29288 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:17.163797 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:36591 (local address 127.26.37.60:45019)
0809 07:53:17.202942 (+ 39145us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:17.202950 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:17.202972 (+ 22us) server_negotiation.cc:408] Connection header received
0809 07:53:17.203037 (+ 65us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:17.203046 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:17.203132 (+ 86us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:17.203255 (+ 123us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:17.221301 (+ 18046us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:36591: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":39019,"thread_start_us":38888,"threads_started":1}
W20260809 07:53:17.224809 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28379: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:35733: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:17.224956 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28379
I20260809 07:53:17.251688 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28572
W20260809 07:53:17.254834 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28572: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:42085: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:17.255004 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28572
I20260809 07:53:17.281044 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28763
W20260809 07:53:17.283749 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28763: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:38157: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:53:17.283908 29291 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:17.282072 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.26.37.62:38157)
0809 07:53:17.282271 (+ 199us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:17.282279 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:17.282366 (+ 87us) server_negotiation.cc:408] Connection header received
0809 07:53:17.282664 (+ 298us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:17.282672 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:17.282751 (+ 79us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:17.282860 (+ 109us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:17.283628 (+ 768us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38622: BlockingRecv error: recv got EOF from 127.0.0.1:38622 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 07:53:17.283962 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28763
W20260809 07:53:17.310492 28044 connection.cc:573] client connection to 127.26.37.62:38157 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:53:17.311084 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27925
I20260809 07:53:17.312386 28006 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 07:53:17.312559 28044 consensus_peers.cc:597] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c -> Peer 074f9dde34844437a15091fcae8153f3 (127.26.37.62:38157): Couldn't send request to peer 074f9dde34844437a15091fcae8153f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:38157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:53:17.324457 28025 master_runner.cc:187] Error getting master registration for 127.26.37.62:38157: Network error: Client connection negotiation failed: client connection to 127.26.37.62:38157: connect: Connection refused (error 111),
I20260809 07:53:17.324708 28025 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:17.424213 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27925
I20260809 07:53:17.455423 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 28025
I20260809 07:53:17.456539 28104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:53:17.567651 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 28025
Shutting down hive metastore.
2026-08-09T07:53:17Z chronyd exiting
Aug 09 07:53:18 dist-test-slave-bw88 krb5kdc[29314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:53:18 dist-test-slave-bw88 krb5kdc[29314](info): set up 2 sockets
Aug 09 07:53:18 dist-test-slave-bw88 krb5kdc[29314](info): commencing operation
krb5kdc: starting...
W20260809 07:53:20.094229 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.111s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:53:20 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:53:20Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:20Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "hive/127.0.0.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:53:21: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:53:23 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262003, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:53:27 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262007, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:53:27 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262007, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:30.048657 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38157
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:30.393869 29569 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:30.394181 29569 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:30.394269 29569 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:30.402346 29569 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:30.402456 29569 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:30.402518 29569 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:30.402565 29569 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:30.413854 29569 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:30.415455 29569 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:30.416908 29569 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:30.425709 29575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:30.425837 29576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:30.426433 29578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:30.427037 29569 server_base.cc:1061] running on GCE node
I20260809 07:53:30.427783 29569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:30.429186 29569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:30.430423 29569 hybrid_clock.cc:648] HybridClock initialized: now 1786262010430361 us; error 57 us; skew 500 ppm
I20260809 07:53:30.430774 29569 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:30 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262010, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:30.434293 29569 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:53:30.435925 29569 webserver.cc:533] Webserver started at http://127.26.37.62:35019/ using document root <none> and password file <none>
I20260809 07:53:30.436682 29569 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:30.436817 29569 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:30.442396 29569 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:53:30.445425 29587 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:30.446748 29569 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:30.446919 29569 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "074f9dde34844437a15091fcae8153f3"
format_stamp: "Formatted at 2026-08-09 07:53:10 on dist-test-slave-bw88"
I20260809 07:53:30.447641 29569 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:30.467515 29569 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:30.476080 29569 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:30.476373 29569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:30.497026 29569 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:38157
I20260809 07:53:30.497045 29656 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:38157 every 8 connection(s)
I20260809 07:53:30.498735 29569 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:53:30.505908 29658 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:53:30.507759 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29569
I20260809 07:53:30.507952 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:53:30.510250 29658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap starting.
I20260809 07:53:30.530578 29658 log.cc:826] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:53:30.540169 29658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:53:30.540817 29658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap complete.
I20260809 07:53:30.551697 29658 raft_consensus.cc:359] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:30.552556 29658 raft_consensus.cc:740] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074f9dde34844437a15091fcae8153f3, State: Initialized, Role: FOLLOWER
I20260809 07:53:30.553469 29658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:30.554426 29664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:30.554703 29664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:30.555585 29658 sys_catalog.cc:565] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:30.567093 29677 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:30.567284 29677 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: loading cluster ID for follower catalog manager: success
I20260809 07:53:30.571913 29677 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: acquiring CA information for follower catalog manager: success
I20260809 07:53:30.575259 29677 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Aug 09 07:53:30 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262010, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:30.596138 29569 master_runner.cc:187] Error getting master registration for 127.26.37.61:42563: Network error: Client connection negotiation failed: client connection to 127.26.37.61:42563: connect: Connection refused (error 111),
I20260809 07:53:30.603274 29569 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:30.645013 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:42563
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:31.000075 29687 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:31.000373 29687 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:31.000453 29687 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:31.008487 29687 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:31.008579 29687 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:31.008632 29687 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:31.008677 29687 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:31.018682 29687 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:42563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:31.020143 29687 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:31.021420 29687 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:31.029973 29696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:31.030046 29697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:31.030875 29699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:31.031586 29687 server_base.cc:1061] running on GCE node
I20260809 07:53:31.032349 29687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:31.033713 29687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:31.034900 29687 hybrid_clock.cc:648] HybridClock initialized: now 1786262011034850 us; error 52 us; skew 500 ppm
I20260809 07:53:31.035243 29687 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:31 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262011, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:31.038743 29687 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:53:31.040234 29687 webserver.cc:533] Webserver started at http://127.26.37.61:40099/ using document root <none> and password file <none>
I20260809 07:53:31.040915 29687 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:31.041028 29687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:31.046357 29687 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:53:31.049010 29709 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:31.050348 29687 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:31.050580 29687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "98e67594146d4876987788c2d89e0b66"
format_stamp: "Formatted at 2026-08-09 07:53:11 on dist-test-slave-bw88"
I20260809 07:53:31.051324 29687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:31.066149 29687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:31.069775 29687 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:31.070021 29687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:31.100723 29687 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:42563
I20260809 07:53:31.100811 29772 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:42563 every 8 connection(s)
I20260809 07:53:31.102443 29687 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:53:31.107579 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29687
I20260809 07:53:31.107841 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:53:31.111056 29773 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:53:31.116251 29773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: Bootstrap starting.
I20260809 07:53:31.135306 29773 log.cc:826] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:53:31.146634 29773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:53:31.147346 29773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: Bootstrap complete.
I20260809 07:53:31.154980 29773 raft_consensus.cc:359] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:31.155670 29773 raft_consensus.cc:740] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e67594146d4876987788c2d89e0b66, State: Initialized, Role: FOLLOWER
I20260809 07:53:31.156659 29773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:31.157582 29778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:31.157933 29778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:31.158754 29773 sys_catalog.cc:565] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:31.170720 29795 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:31.170831 29795 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: loading cluster ID for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:31.174926 29795 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: acquiring CA information for follower catalog manager: success
I20260809 07:53:31.177762 29795 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:53:31 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262011, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:53:31.214102 29687 master_runner.cc:187] Error getting master registration for 127.26.37.60:45019: Network error: Client connection negotiation failed: client connection to 127.26.37.60:45019: connect: Connection refused (error 111),
I20260809 07:53:31.219758 29687 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:31.240656 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:45019
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:31.595103 29803 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:31.595428 29803 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:31.595593 29803 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:31.603621 29803 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:31.603703 29803 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:31.603806 29803 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:31.603881 29803 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:31.613890 29803 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:45019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:31.615453 29803 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:31.616807 29803 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:31.624655 29810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:31.624686 29809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:31.625587 29813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:31.625962 29803 server_base.cc:1061] running on GCE node
I20260809 07:53:31.626699 29803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:31.627997 29803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:31.629220 29803 hybrid_clock.cc:648] HybridClock initialized: now 1786262011629189 us; error 57 us; skew 500 ppm
I20260809 07:53:31.629587 29803 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:31 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262011, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:31.632835 29803 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:53:31.634505 29803 webserver.cc:533] Webserver started at http://127.26.37.60:44517/ using document root <none> and password file <none>
I20260809 07:53:31.635241 29803 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:31.635396 29803 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:31.640640 29803 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 07:53:31.643458 29820 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:31.644805 29803 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:53:31.644994 29803 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "f693852090fa40b69fc88f7d1366c47c"
format_stamp: "Formatted at 2026-08-09 07:53:12 on dist-test-slave-bw88"
I20260809 07:53:31.645792 29803 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:31.684613 29803 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:31.688170 29803 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:31.688452 29803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:31.709084 29803 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:45019
I20260809 07:53:31.709123 29886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:45019 every 8 connection(s)
I20260809 07:53:31.710776 29803 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:53:31.717281 29887 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:53:31.720521 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Bootstrap starting.
I20260809 07:53:31.720602 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29803
I20260809 07:53:31.720852 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
Aug 09 07:53:31 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:31.741426 29887 log.cc:826] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:53:31 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:31.751614 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:53:31.752233 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Bootstrap complete.
I20260809 07:53:31.761530 29887 raft_consensus.cc:359] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:31.762246 29887 raft_consensus.cc:740] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f693852090fa40b69fc88f7d1366c47c, State: Initialized, Role: FOLLOWER
I20260809 07:53:31.763021 29887 consensus_queue.cc:260] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:31.764305 29895 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:31.764734 29895 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:31.765344 29887 sys_catalog.cc:565] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 07:53:31 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:53:31 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262011, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:31.784622 29908 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:31.784739 29908 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: loading cluster ID for follower catalog manager: success
I20260809 07:53:31.788364 29908 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: acquiring CA information for follower catalog manager: success
I20260809 07:53:31.790745 29908 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:53:31.810850 29803 master_runner.cc:264] No leader master found from master f693852090fa40b69fc88f7d1366c47c
I20260809 07:53:31.810989 29803 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:32.406234 29911 raft_consensus.cc:493] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:53:32.406548 29911 raft_consensus.cc:515] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:32.408326 29911 leader_election.cc:290] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98e67594146d4876987788c2d89e0b66 (127.26.37.61:42563), f693852090fa40b69fc88f7d1366c47c (127.26.37.60:45019)
I20260809 07:53:32.427378 29742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "074f9dde34844437a15091fcae8153f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "98e67594146d4876987788c2d89e0b66" is_pre_election: true
I20260809 07:53:32.427817 29855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "074f9dde34844437a15091fcae8153f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f693852090fa40b69fc88f7d1366c47c" is_pre_election: true
I20260809 07:53:32.428035 29742 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 074f9dde34844437a15091fcae8153f3 in term 1.
I20260809 07:53:32.428385 29855 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 074f9dde34844437a15091fcae8153f3 in term 1.
I20260809 07:53:32.428752 29589 leader_election.cc:304] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074f9dde34844437a15091fcae8153f3, 98e67594146d4876987788c2d89e0b66; no voters:
I20260809 07:53:32.429162 29911 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:53:32.429325 29911 raft_consensus.cc:493] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:53:32.429466 29911 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:53:32.433135 29911 raft_consensus.cc:515] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:32.433847 29911 leader_election.cc:290] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [CANDIDATE]: Term 2 election: Requested vote from peers 98e67594146d4876987788c2d89e0b66 (127.26.37.61:42563), f693852090fa40b69fc88f7d1366c47c (127.26.37.60:45019)
I20260809 07:53:32.434177 29742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "074f9dde34844437a15091fcae8153f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "98e67594146d4876987788c2d89e0b66"
I20260809 07:53:32.434324 29855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "074f9dde34844437a15091fcae8153f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f693852090fa40b69fc88f7d1366c47c"
I20260809 07:53:32.434399 29742 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:53:32.434552 29855 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:53:32.437992 29742 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 074f9dde34844437a15091fcae8153f3 in term 2.
I20260809 07:53:32.438015 29855 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 074f9dde34844437a15091fcae8153f3 in term 2.
I20260809 07:53:32.438436 29589 leader_election.cc:304] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074f9dde34844437a15091fcae8153f3, 98e67594146d4876987788c2d89e0b66; no voters:
I20260809 07:53:32.438789 29911 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:53:32.439200 29911 raft_consensus.cc:697] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 2 LEADER]: Becoming Leader. State: Replica: 074f9dde34844437a15091fcae8153f3, State: Running, Role: LEADER
I20260809 07:53:32.439639 29911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:32.442440 29917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 074f9dde34844437a15091fcae8153f3. Latest consensus state: current_term: 2 leader_uuid: "074f9dde34844437a15091fcae8153f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:32.442883 29917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:32.443611 29921 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:53:32.445557 29921 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:53:32.446311 29921 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:32.446415 29921 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:53:32.447413 29921 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:53:32.448160 29921 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Loaded TSK: 0
I20260809 07:53:32.448997 29921 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:32.602617 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:32.922055 29855 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 2 FOLLOWER]: Refusing update from remote peer 074f9dde34844437a15091fcae8153f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:53:32.923221 29917 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:53:32.927443 29928 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 074f9dde34844437a15091fcae8153f3. Latest consensus state: current_term: 2 leader_uuid: "074f9dde34844437a15091fcae8153f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:32.927735 29928 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:32.929143 29911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "074f9dde34844437a15091fcae8153f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:32.929476 29911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:32.931222 29742 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 2 FOLLOWER]: Refusing update from remote peer 074f9dde34844437a15091fcae8153f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:53:32.932369 29911 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:53:32.944586 29928 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "074f9dde34844437a15091fcae8153f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:32.944878 29928 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:32.947134 29936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "074f9dde34844437a15091fcae8153f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:32.947428 29936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:32.947885 29933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 074f9dde34844437a15091fcae8153f3. Latest consensus state: current_term: 2 leader_uuid: "074f9dde34844437a15091fcae8153f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:32.948172 29933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:53:32.961969 29927 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:32.962339 29927 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:32.962512 29927 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:32.970801 29927 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:32.971040 29927 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:32.981528 29927 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:32.983176 29927 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:32.984570 29927 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:32.994170 29945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:32.994184 29944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:32.995067 29947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:32.995514 29927 server_base.cc:1061] running on GCE node
I20260809 07:53:32.996044 29927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:32.996907 29927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:32.998139 29927 hybrid_clock.cc:648] HybridClock initialized: now 1786262012998105 us; error 56 us; skew 500 ppm
I20260809 07:53:32.998479 29927 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:32 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262012, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:33.002307 29927 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:33.003875 29927 webserver.cc:533] Webserver started at http://127.26.37.1:39877/ using document root <none> and password file <none>
I20260809 07:53:33.004670 29927 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:33.004809 29927 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:33.010486 29927 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:53:33.013232 29955 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:33.014577 29927 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:33.014752 29927 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:33.015491 29927 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:33.032661 29927 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:33.036580 29927 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:33.036881 29927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:33.037940 29927 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:33.039628 29927 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:33.039783 29927 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:33.039906 29927 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:33.039994 29927 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:33.076316 29927 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44099
I20260809 07:53:33.076406 30101 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44099 every 8 connection(s)
I20260809 07:53:33.077960 29927 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:53:33.086055 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 29927
I20260809 07:53:33.086305 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:53:33 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262012, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:33.115214 30105 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42563
I20260809 07:53:33.115723 30105 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:33.117143 30105 heartbeater.cc:507] Master 127.26.37.61:42563 requested a full tablet report, sending...
I20260809 07:53:33.121455 29732 ts_manager.cc:194] Registered new tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:44099)
Aug 09 07:53:33 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262012, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:33 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262012, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:33.136701 30102 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:33.136878 30102 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:33.137444 30102 heartbeater.cc:507] Master 127.26.37.62:38157 requested a full tablet report, sending...
I20260809 07:53:33.139477 29610 ts_manager.cc:194] Registered new tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:44099)
I20260809 07:53:33.139701 30103 heartbeater.cc:344] Connected to a master server at 127.26.37.60:45019
I20260809 07:53:33.139825 30103 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:33.140246 30103 heartbeater.cc:507] Master 127.26.37.60:45019 requested a full tablet report, sending...
I20260809 07:53:33.141916 29840 ts_manager.cc:194] Registered new tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:44099)
I20260809 07:53:33.142104 29610 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:36087
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:33.241984 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:33.339051 29931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "074f9dde34844437a15091fcae8153f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:33.339402 29931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: LEADER
W20260809 07:53:33.582252 30120 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:33.582618 30120 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:33.582765 30120 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:33.591084 30120 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:33.591305 30120 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:53:33.601336 30120 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:33.602973 30120 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:33.604301 30120 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:33.613873 30126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:33.614485 30127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:33.614854 30129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:33.615259 30120 server_base.cc:1061] running on GCE node
I20260809 07:53:33.615792 30120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:33.616637 30120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:33.617871 30120 hybrid_clock.cc:648] HybridClock initialized: now 1786262013617835 us; error 45 us; skew 500 ppm
I20260809 07:53:33.618162 30120 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:33 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262013, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:33.621769 30120 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:53:33.623261 30120 webserver.cc:533] Webserver started at http://127.26.37.2:35345/ using document root <none> and password file <none>
I20260809 07:53:33.623914 30120 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:33.624020 30120 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:33.629122 30120 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:53:33.631920 30136 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:33.633280 30120 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:53:33.633505 30120 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "6a8f8e269e6d40c0bc2dd545a0ecdcc8"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:33.634187 30120 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:33.664588 30120 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:33.668531 30120 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:33.668870 30120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:33.669987 30120 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:33.671696 30120 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:33.671787 30120 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:33.671936 30120 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:33.672031 30120 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:33.707235 30120 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:41441
I20260809 07:53:33.707324 30286 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:41441 every 8 connection(s)
I20260809 07:53:33.708977 30120 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:53:33.719420 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30120
I20260809 07:53:33.719640 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:53:33 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262013, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:33 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262013, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:33.764855 30289 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:33.765323 30289 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:33.766458 30289 heartbeater.cc:507] Master 127.26.37.62:38157 requested a full tablet report, sending...
I20260809 07:53:33.768529 29610 ts_manager.cc:194] Registered new tserver with Master: 6a8f8e269e6d40c0bc2dd545a0ecdcc8 (127.26.37.2:41441)
I20260809 07:53:33.769721 29610 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:46463
Aug 09 07:53:33 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262013, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:33.772524 30290 heartbeater.cc:344] Connected to a master server at 127.26.37.60:45019
I20260809 07:53:33.772694 30290 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:33.773126 30290 heartbeater.cc:507] Master 127.26.37.60:45019 requested a full tablet report, sending...
I20260809 07:53:33.774186 29840 ts_manager.cc:194] Registered new tserver with Master: 6a8f8e269e6d40c0bc2dd545a0ecdcc8 (127.26.37.2:41441)
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:53:33.779495 30291 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42563
I20260809 07:53:33.779635 30291 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:33.779865 30291 heartbeater.cc:507] Master 127.26.37.61:42563 requested a full tablet report, sending...
I20260809 07:53:33.780766 29732 ts_manager.cc:194] Registered new tserver with Master: 6a8f8e269e6d40c0bc2dd545a0ecdcc8 (127.26.37.2:41441)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:33.869460 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:34.145692 30102 heartbeater.cc:499] Master 127.26.37.62:38157 was elected leader, sending a full tablet report...
W20260809 07:53:34.217552 30311 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:34.217887 30311 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:34.218030 30311 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:34.226166 30311 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:34.226372 30311 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:53:34.236570 30311 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:34.238293 30311 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:34.239753 30311 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:34.249665 30317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:34.249702 30316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:34.250268 30319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:34.251154 30311 server_base.cc:1061] running on GCE node
I20260809 07:53:34.251766 30311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:34.252732 30311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:34.253975 30311 hybrid_clock.cc:648] HybridClock initialized: now 1786262014253925 us; error 39 us; skew 500 ppm
I20260809 07:53:34.254308 30311 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:34 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262014, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:34.258385 30311 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:53:34.260224 30311 webserver.cc:533] Webserver started at http://127.26.37.3:41809/ using document root <none> and password file <none>
I20260809 07:53:34.260931 30311 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:34.261144 30311 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:34.266553 30311 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:53:34.269208 30328 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:34.270608 30311 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:34.270792 30311 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "a9d033442a3f4232b275f79e9f1a1dde"
format_stamp: "Formatted at 2026-08-09 07:53:14 on dist-test-slave-bw88"
I20260809 07:53:34.271725 30311 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:34.306651 30311 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:34.310706 30311 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:34.310974 30311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:34.311888 30311 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:34.313401 30311 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:34.313495 30311 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:34.313576 30311 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:34.313621 30311 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:34.349954 30311 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:45401
I20260809 07:53:34.350000 30473 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:45401 every 8 connection(s)
I20260809 07:53:34.351536 30311 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:53:34.358559 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30311
I20260809 07:53:34.358790 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:53:34 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262014, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:34 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262014, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:53:34 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262014, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:34.397728 30474 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:34.397871 30475 heartbeater.cc:344] Connected to a master server at 127.26.37.60:45019
I20260809 07:53:34.398141 30475 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:34.398150 30474 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:34.398808 30476 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42563
I20260809 07:53:34.398972 30476 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:34.399312 30475 heartbeater.cc:507] Master 127.26.37.60:45019 requested a full tablet report, sending...
I20260809 07:53:34.399363 30474 heartbeater.cc:507] Master 127.26.37.62:38157 requested a full tablet report, sending...
I20260809 07:53:34.399546 30476 heartbeater.cc:507] Master 127.26.37.61:42563 requested a full tablet report, sending...
I20260809 07:53:34.401413 29732 ts_manager.cc:194] Registered new tserver with Master: a9d033442a3f4232b275f79e9f1a1dde (127.26.37.3:45401)
I20260809 07:53:34.401561 29610 ts_manager.cc:194] Registered new tserver with Master: a9d033442a3f4232b275f79e9f1a1dde (127.26.37.3:45401)
I20260809 07:53:34.401669 29841 ts_manager.cc:194] Registered new tserver with Master: a9d033442a3f4232b275f79e9f1a1dde (127.26.37.3:45401)
I20260809 07:53:34.402645 29610 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:44383
I20260809 07:53:34.417143 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:53:34.424041 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:53:34.424220 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29569
I20260809 07:53:34.425043 29650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:53:34.565891 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29569
I20260809 07:53:34.595252 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38157
--webserver_interface=127.26.37.62
--webserver_port=35019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:34.773238 30289 heartbeater.cc:499] Master 127.26.37.62:38157 was elected leader, sending a full tablet report...
W20260809 07:53:34.775007 30289 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:38157 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:38157: connect: Connection refused (error 111)
W20260809 07:53:34.958390 30488 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:34.958755 30488 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:34.958905 30488 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:34.967103 30488 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:34.967190 30488 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:34.967329 30488 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:34.967406 30488 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:34.978003 30488 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=35019
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:34.979784 30488 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:34.981137 30488 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:34.990036 30495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:34.990113 30496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:34.990630 30498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:34.991444 30488 server_base.cc:1061] running on GCE node
I20260809 07:53:34.992197 30488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:34.993588 30488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:34.994835 30488 hybrid_clock.cc:648] HybridClock initialized: now 1786262014994775 us; error 56 us; skew 500 ppm
I20260809 07:53:34.995189 30488 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:34 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262014, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:34.998935 30488 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:53:35.000535 30488 webserver.cc:533] Webserver started at http://127.26.37.62:35019/ using document root <none> and password file <none>
I20260809 07:53:35.001289 30488 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:35.001452 30488 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:35.006907 30488 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 07:53:35.009749 30510 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:35.011116 30488 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:35.011307 30488 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "074f9dde34844437a15091fcae8153f3"
format_stamp: "Formatted at 2026-08-09 07:53:10 on dist-test-slave-bw88"
I20260809 07:53:35.012018 30488 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:35.029168 30488 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:35.032989 30488 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:35.033198 30488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:35.054723 30488 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:38157
I20260809 07:53:35.054745 30575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:38157 every 8 connection(s)
I20260809 07:53:35.056387 30488 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:53:35.056758 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30488
I20260809 07:53:35.057133 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29927
I20260809 07:53:35.064226 30580 sys_catalog.cc:263] Verifying existing consensus state
Aug 09 07:53:35 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:53:35.068136 30580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap starting.
I20260809 07:53:35.076189 30091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:53:35.090062 30580 log.cc:826] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:35.099870 30580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:53:35.100411 30580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: Bootstrap complete.
I20260809 07:53:35.107527 30580 raft_consensus.cc:359] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:35.108189 30580 raft_consensus.cc:740] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074f9dde34844437a15091fcae8153f3, State: Initialized, Role: FOLLOWER
I20260809 07:53:35.108952 30580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:35.109839 30588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:35.110201 30588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:35.110913 30580 sys_catalog.cc:565] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:35.134163 30608 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:35.134330 30608 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: loading cluster ID for follower catalog manager: success
I20260809 07:53:35.140035 30608 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: acquiring CA information for follower catalog manager: success
I20260809 07:53:35.142918 30608 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:53:35 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262014, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:35.177265 30488 master_runner.cc:264] No leader master found from master 074f9dde34844437a15091fcae8153f3
I20260809 07:53:35.177475 30488 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:35.208842 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29927
W20260809 07:53:35.241566 29823 connection.cc:573] server connection from 127.26.37.1:42197 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:53:35.243048 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44099
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--webserver_port=39877
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:35.243254 30612 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:35.205806 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:52623 (local address 127.26.37.62:38157)
0809 07:53:35.206205 (+ 399us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:35.206215 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:35.213079 (+ 6864us) server_negotiation.cc:408] Connection header received
0809 07:53:35.213173 (+ 94us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:35.213179 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:35.213261 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:35.213398 (+ 137us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0809 07:53:35.241802 (+ 28404us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:52623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 07:53:35.405097 30474 heartbeater.cc:499] Master 127.26.37.62:38157 was elected leader, sending a full tablet report...
I20260809 07:53:35.415928 30533 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a9d033442a3f4232b275f79e9f1a1dde" instance_seqno: 1786262014341676) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:58033; Asking this server to re-register.
I20260809 07:53:35.416882 30474 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:35.418676 30533 ts_manager.cc:194] Registered new tserver with Master: a9d033442a3f4232b275f79e9f1a1dde (127.26.37.3:45401)
W20260809 07:53:35.599107 30617 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:35.599421 30617 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:35.599525 30617 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:35.608292 30617 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:35.608497 30617 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:35.620236 30617 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39877
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:35.621971 30617 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:35.623308 30617 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:35.633021 30626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:35.633256 30627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:35.634661 30629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:35.635114 30617 server_base.cc:1061] running on GCE node
I20260809 07:53:35.635723 30617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:35.636777 30617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:35.638018 30617 hybrid_clock.cc:648] HybridClock initialized: now 1786262015637958 us; error 68 us; skew 500 ppm
I20260809 07:53:35.638355 30617 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:35 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262015, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:35.642059 30617 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:35.643755 30617 webserver.cc:533] Webserver started at http://127.26.37.1:39877/ using document root <none> and password file <none>
I20260809 07:53:35.644558 30617 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:35.644702 30617 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:35.650321 30617 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:53:35.653147 30640 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:35.654749 30617 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:53:35.654932 30617 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:35.655671 30617 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:35.680191 30617 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:35.706353 30617 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:35.706719 30617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:35.707824 30617 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:35.709765 30617 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:35.709877 30617 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:35.709981 30617 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:35.710064 30617 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:35.713778 30652 raft_consensus.cc:493] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 074f9dde34844437a15091fcae8153f3)
I20260809 07:53:35.713985 30652 raft_consensus.cc:515] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:35.715894 30652 leader_election.cc:290] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 074f9dde34844437a15091fcae8153f3 (127.26.37.62:38157), 98e67594146d4876987788c2d89e0b66 (127.26.37.61:42563)
I20260809 07:53:35.717219 29742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f693852090fa40b69fc88f7d1366c47c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "98e67594146d4876987788c2d89e0b66" is_pre_election: true
I20260809 07:53:35.725107 30546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f693852090fa40b69fc88f7d1366c47c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "074f9dde34844437a15091fcae8153f3" is_pre_election: true
I20260809 07:53:35.725831 30546 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f693852090fa40b69fc88f7d1366c47c in term 2.
I20260809 07:53:35.726646 29821 leader_election.cc:304] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 074f9dde34844437a15091fcae8153f3, f693852090fa40b69fc88f7d1366c47c; no voters: 98e67594146d4876987788c2d89e0b66
I20260809 07:53:35.727087 30652 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:53:35.727243 30652 raft_consensus.cc:493] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 074f9dde34844437a15091fcae8153f3)
I20260809 07:53:35.727377 30652 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:53:35.730486 30652 raft_consensus.cc:515] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:35.731694 30546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f693852090fa40b69fc88f7d1366c47c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "074f9dde34844437a15091fcae8153f3"
I20260809 07:53:35.731803 29742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f693852090fa40b69fc88f7d1366c47c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "98e67594146d4876987788c2d89e0b66"
I20260809 07:53:35.731935 30652 leader_election.cc:290] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [CANDIDATE]: Term 3 election: Requested vote from peers 074f9dde34844437a15091fcae8153f3 (127.26.37.62:38157), 98e67594146d4876987788c2d89e0b66 (127.26.37.61:42563)
I20260809 07:53:35.731921 30546 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:53:35.738756 30546 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f693852090fa40b69fc88f7d1366c47c in term 3.
I20260809 07:53:35.739326 29821 leader_election.cc:304] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 074f9dde34844437a15091fcae8153f3, f693852090fa40b69fc88f7d1366c47c; no voters: 98e67594146d4876987788c2d89e0b66
I20260809 07:53:35.739703 30652 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:53:35.740183 30652 raft_consensus.cc:697] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [term 3 LEADER]: Becoming Leader. State: Replica: f693852090fa40b69fc88f7d1366c47c, State: Running, Role: LEADER
I20260809 07:53:35.740689 30652 consensus_queue.cc:237] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
I20260809 07:53:35.742914 30731 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f693852090fa40b69fc88f7d1366c47c. Latest consensus state: current_term: 3 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:35.743315 30731 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:35.744082 30746 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:53:35.746721 30746 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:53:35.747859 30746 catalog_manager.cc:1295] Loaded cluster ID: 6164785b958e44c0b1117618838b450b
I20260809 07:53:35.748010 30746 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:53:35.749634 30746 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:53:35.750514 30746 catalog_manager.cc:6103] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c: Loaded TSK: 0
I20260809 07:53:35.751444 30746 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:53:35.758970 30617 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44099
I20260809 07:53:35.759053 30798 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44099 every 8 connection(s)
I20260809 07:53:35.760577 30617 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:53:35.766126 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30617
Aug 09 07:53:35 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:53:35 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262015, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:35 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262015, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:35.795468 30813 raft_consensus.cc:493] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 074f9dde34844437a15091fcae8153f3)
I20260809 07:53:35.796373 30813 raft_consensus.cc:515] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } }
Aug 09 07:53:35 dist-test-slave-bw88 krb5kdc[29314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262015, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:35.802786 30799 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:35.803218 30799 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:35.804343 30799 heartbeater.cc:507] Master 127.26.37.62:38157 requested a full tablet report, sending...
I20260809 07:53:35.806684 30533 ts_manager.cc:194] Registered new tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:44099)
I20260809 07:53:35.816576 30804 heartbeater.cc:344] Connected to a master server at 127.26.37.61:42563
I20260809 07:53:35.816740 30804 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:35.817288 30804 heartbeater.cc:507] Master 127.26.37.61:42563 requested a full tablet report, sending...
I20260809 07:53:35.818887 29732 ts_manager.cc:194] Re-registered known tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:44099)
I20260809 07:53:35.820137 30802 heartbeater.cc:344] Connected to a master server at 127.26.37.60:45019
I20260809 07:53:35.820277 30802 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:35.820750 30802 heartbeater.cc:507] Master 127.26.37.60:45019 requested a full tablet report, sending...
I20260809 07:53:35.821426 30813 leader_election.cc:290] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 074f9dde34844437a15091fcae8153f3 (127.26.37.62:38157), f693852090fa40b69fc88f7d1366c47c (127.26.37.60:45019)
I20260809 07:53:35.822018 29841 ts_manager.cc:194] Re-registered known tserver with Master: 345673922d414239be337630f1b89537 (127.26.37.1:44099)
I20260809 07:53:35.823868 29841 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:37177
I20260809 07:53:35.824291 30289 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38157
I20260809 07:53:35.824465 30289 heartbeater.cc:499] Master 127.26.37.62:38157 was elected leader, sending a full tablet report...
I20260809 07:53:35.825507 30533 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6a8f8e269e6d40c0bc2dd545a0ecdcc8" instance_seqno: 1786262013698845) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:35421; Asking this server to re-register.
I20260809 07:53:35.826195 30289 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:35.827303 30533 ts_manager.cc:194] Registered new tserver with Master: 6a8f8e269e6d40c0bc2dd545a0ecdcc8 (127.26.37.2:41441)
I20260809 07:53:35.833596 29855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98e67594146d4876987788c2d89e0b66" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f693852090fa40b69fc88f7d1366c47c" is_pre_election: true
I20260809 07:53:35.836145 30546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98e67594146d4876987788c2d89e0b66" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "074f9dde34844437a15091fcae8153f3" is_pre_election: true
I20260809 07:53:35.836544 30546 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98e67594146d4876987788c2d89e0b66 in current term 3: Already voted for candidate f693852090fa40b69fc88f7d1366c47c in this term.
I20260809 07:53:35.837312 29710 leader_election.cc:304] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98e67594146d4876987788c2d89e0b66; no voters: 074f9dde34844437a15091fcae8153f3, f693852090fa40b69fc88f7d1366c47c
I20260809 07:53:35.837817 30813 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:53:35.840994 30813 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:53:35.931666 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30617
W20260809 07:53:35.932981 30826 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:35.929115 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:52746 (local address 127.26.37.1:44099)
0809 07:53:35.929829 (+ 714us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:35.929836 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:35.929869 (+ 33us) server_negotiation.cc:408] Connection header received
0809 07:53:35.930041 (+ 172us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:35.930050 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:35.930278 (+ 228us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:35.930459 (+ 181us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:35.931543 (+ 1084us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52746: BlockingRecv error: recv got EOF from 127.0.0.1:52746 (error 108)
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":387,"threads_started":1}
W20260809 07:53:35.934182 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30617: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:44099: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:35.934341 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30617
I20260809 07:53:35.960614 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44099
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--webserver_port=39877
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:36.271833 29742 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [term 3 FOLLOWER]: Refusing update from remote peer f693852090fa40b69fc88f7d1366c47c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:53:36.272651 30830 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [LEADER]: Connected to new peer: Peer: permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:53:36.274614 30813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f693852090fa40b69fc88f7d1366c47c. Latest consensus state: current_term: 3 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:36.274888 30813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:36.277129 30834 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:36.277446 30834 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:36.278376 30546 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [term 3 FOLLOWER]: Refusing update from remote peer f693852090fa40b69fc88f7d1366c47c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:53:36.279575 30834 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [LEADER]: Connected to new peer: Peer: permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:53:36.280791 30813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:36.281008 30813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e67594146d4876987788c2d89e0b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:36.284709 30844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:36.285120 30844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:36.285004 30840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f693852090fa40b69fc88f7d1366c47c. Latest consensus state: current_term: 3 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:36.285319 30840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 074f9dde34844437a15091fcae8153f3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:53:36.323371 30829 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:36.323679 30829 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:36.323778 30829 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:53:36.323834 30829 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:36.332018 30829 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:36.332221 30829 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:36.342562 30829 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39877
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:36.344065 30829 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:36.345499 30829 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:36.354938 30852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:36.355325 30851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:36.355880 30854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:36.356745 30829 server_base.cc:1061] running on GCE node
I20260809 07:53:36.357326 30829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:36.358266 30829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:36.359556 30829 hybrid_clock.cc:648] HybridClock initialized: now 1786262016359488 us; error 34 us; skew 500 ppm
I20260809 07:53:36.359836 30829 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:36 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262016, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:36.363520 30829 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:36.364967 30829 webserver.cc:533] Webserver started at http://127.26.37.1:39877/ using document root <none> and password file <none>
I20260809 07:53:36.365782 30829 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:36.365908 30829 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:36.371300 30829 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 07:53:36.373895 30861 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:36.375207 30829 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:53:36.375376 30829 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:36.376049 30829 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:36.388970 30829 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:36.392871 30829 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:36.393163 30829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:36.394129 30829 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:36.395718 30829 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:36.395823 30829 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:36.395977 30829 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:36.396052 30829 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:53:36.396569 30829 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:53:36.396658 30829 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:53:36.396755 30829 fault_injection.cc:59] injecting fault for kudu (pid 30829): FLAGS_fault_before_start (process will exit)
I20260809 07:53:36.422971 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44099
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--webserver_port=39877
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:36.779551 30290 heartbeater.cc:499] Master 127.26.37.60:45019 was elected leader, sending a full tablet report...
W20260809 07:53:36.779593 30870 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:36.780107 30870 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:36.780293 30870 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:36.788717 30870 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:36.788939 30870 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:36.799320 30870 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39877
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38157,127.26.37.61:42563,127.26.37.60:45019
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:36.800973 30870 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:36.802537 30870 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:36.812333 30876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:36.812906 30875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:36.813438 30878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:36.813796 30870 server_base.cc:1061] running on GCE node
I20260809 07:53:36.814424 30870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:36.815536 30870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:36.816788 30870 hybrid_clock.cc:648] HybridClock initialized: now 1786262016816739 us; error 39 us; skew 500 ppm
I20260809 07:53:36.817130 30870 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:36 dist-test-slave-bw88 krb5kdc[29314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262016, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:36.821051 30870 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:36.822687 30870 webserver.cc:533] Webserver started at http://127.26.37.1:39877/ using document root <none> and password file <none>
I20260809 07:53:36.823395 30870 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:36.823524 30870 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:36.829025 30870 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 07:53:36.831772 30890 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:36.833055 30870 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:53:36.833211 30870 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "345673922d414239be337630f1b89537"
format_stamp: "Formatted at 2026-08-09 07:53:13 on dist-test-slave-bw88"
I20260809 07:53:36.833912 30870 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:36.844292 30885 sys_catalog.cc:455] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f693852090fa40b69fc88f7d1366c47c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074f9dde34844437a15091fcae8153f3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38157 } } peers { permanent_uuid: "98e67594146d4876987788c2d89e0b66" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 42563 } } peers { permanent_uuid: "f693852090fa40b69fc88f7d1366c47c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 45019 } } }
I20260809 07:53:36.844506 30885 sys_catalog.cc:458] T 00000000000000000000000000000000 P f693852090fa40b69fc88f7d1366c47c [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:36.853698 30870 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:36.857786 30870 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:36.858064 30870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:36.858986 30870 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:36.860617 30870 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:36.860726 30870 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:36.860862 30870 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:36.860939 30870 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:36.894021 30870 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44099
I20260809 07:53:36.894064 31033 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44099 every 8 connection(s)
I20260809 07:53:36.895526 30870 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:53:36.895983 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 30870
I20260809 07:53:36.896355 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30870
W20260809 07:53:36.916292 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30870: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:44099: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:36.916492 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30870
W20260809 07:53:36.936563 31044 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:36.908702 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:50617 (local address 127.26.37.62:38157)
0809 07:53:36.909804 (+ 1102us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:36.909811 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:36.914898 (+ 5087us) server_negotiation.cc:408] Connection header received
0809 07:53:36.916351 (+ 1453us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:36.916371 (+ 20us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:36.916458 (+ 87us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:36.916572 (+ 114us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:36.936214 (+ 19642us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:50617: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":992,"thread_start_us":875,"threads_started":1}
W20260809 07:53:36.937147 31039 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:36.898608 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:58347 (local address 127.26.37.60:45019)
0809 07:53:36.913428 (+ 14820us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:36.913436 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:36.913469 (+ 33us) server_negotiation.cc:408] Connection header received
0809 07:53:36.917630 (+ 4161us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:36.917641 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:36.917753 (+ 112us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:36.917892 (+ 139us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:36.936254 (+ 18362us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:58347: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":14640,"thread_start_us":14489,"threads_started":1}
I20260809 07:53:36.937156 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30120
W20260809 07:53:36.937937 31048 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:36.915238 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:58191 (local address 127.26.37.61:42563)
0809 07:53:36.918514 (+ 3276us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:36.918532 (+ 18us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:36.937182 (+ 18650us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:58191: BlockingRecv error: recv got EOF from 127.26.37.1:58191 (error 108)
Metrics: {"server-negotiator.queue_time_us":3162,"thread_start_us":3014,"threads_started":1}
W20260809 07:53:36.939985 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30120: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:41441: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:36.940164 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30120
I20260809 07:53:36.962267 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30311
W20260809 07:53:36.964860 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30311: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:45401: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:36.965034 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30311
I20260809 07:53:36.986160 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30488
W20260809 07:53:36.988229 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 30488: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:38157: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:53:36.988394 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 30488
W20260809 07:53:36.988469 31044 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:53:36.986998 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.26.37.62:38157)
0809 07:53:36.987201 (+ 203us) server_negotiation.cc:207] Beginning negotiation
0809 07:53:36.987206 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:53:36.987223 (+ 17us) server_negotiation.cc:408] Connection header received
0809 07:53:36.987356 (+ 133us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:53:36.987361 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:53:36.987424 (+ 63us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:53:36.987506 (+ 82us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:53:36.988203 (+ 697us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47274: BlockingRecv error: recv got EOF from 127.0.0.1:47274 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 07:53:37.018463 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29687
I20260809 07:53:37.019695 29768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:53:37.132380 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29687
I20260809 07:53:37.162308 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 29803
I20260809 07:53:37.163476 29881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:53:37.274456 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 29803
Shutting down hive metastore.
2026-08-09T07:53:37Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/10 (46561 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/11
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:53:37 dist-test-slave-bw88 krb5kdc[31063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:53:37 dist-test-slave-bw88 krb5kdc[31063](info): set up 2 sockets
Aug 09 07:53:37 dist-test-slave-bw88 krb5kdc[31063](info): commencing operation
krb5kdc: starting...
W20260809 07:53:39.896812 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.186s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:53:40 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:53:40Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:40Z Disabled control of system clock
2026-08-09T07:53:40Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:40Z Disabled control of system clock
2026-08-09T07:53:40Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:40Z Disabled control of system clock
2026-08-09T07:53:40Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:40Z Disabled control of system clock
2026-08-09T07:53:40Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:40Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:53:47: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:53:49 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262029, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:53:55 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262035, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:53:55 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262035, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:56.632517 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33383
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:56.971917 31444 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:56.972276 31444 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:56.972417 31444 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:56.980149 31444 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:56.980293 31444 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:56.980402 31444 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:56.980479 31444 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:56.990597 31444 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:56.992231 31444 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:56.993908 31444 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:57.001560 31451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:57.002621 31452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:57.002671 31455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:57.003918 31444 server_base.cc:1061] running on GCE node
I20260809 07:53:57.004599 31444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:57.006443 31444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:57.007678 31444 hybrid_clock.cc:648] HybridClock initialized: now 1786262037007538 us; error 109 us; skew 500 ppm
I20260809 07:53:57.007989 31444 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:57 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262037, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:57.011260 31444 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:53:57.012779 31444 webserver.cc:533] Webserver started at http://127.26.37.62:46753/ using document root <none> and password file <none>
I20260809 07:53:57.013590 31444 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:57.013739 31444 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:57.014117 31444 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:57.016549 31444 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "943334b0c66e48a7b966b6623474b72f"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:53:57.017431 31444 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "943334b0c66e48a7b966b6623474b72f"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:53:57.022818 31444 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:53:57.026099 31464 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:57.027509 31444 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:53:57.027716 31444 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "943334b0c66e48a7b966b6623474b72f"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:53:57.027916 31444 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:57.043802 31444 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:57.047709 31444 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:57.048043 31444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:57.070246 31444 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33383
I20260809 07:53:57.070292 31532 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33383 every 8 connection(s)
I20260809 07:53:57.071866 31444 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:53:57.076432 31533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:57.081780 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31444
I20260809 07:53:57.082021 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:53:57.082384 31533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:57 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262037, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:57.113476 31533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:53:57.114699 31465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39893: connect: Connection refused (error 111)
W20260809 07:53:57.116358 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39893: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39893: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:57.165731 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } attempt: 1
W20260809 07:53:57.168023 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39893: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39893: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:57.216346 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39893
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:57.236318 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } attempt: 2
W20260809 07:53:57.239342 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39893: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39893: connect: Connection refused (error 111)
I20260809 07:53:57.330629 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } attempt: 3
W20260809 07:53:57.332846 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39893: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39893: connect: Connection refused (error 111)
I20260809 07:53:57.476136 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } attempt: 4
W20260809 07:53:57.478422 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39893: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39893: connect: Connection refused (error 111)
W20260809 07:53:57.561718 31549 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:57.562040 31549 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:57.562215 31549 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:57.571056 31549 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:57.571153 31549 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:57.571249 31549 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:57.571336 31549 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:57.581954 31549 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:57.583609 31549 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:57.585013 31549 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:57.593158 31556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:57.593092 31555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:57.594244 31559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:57.594329 31549 server_base.cc:1061] running on GCE node
I20260809 07:53:57.595112 31549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:57.596634 31549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:57.597846 31549 hybrid_clock.cc:648] HybridClock initialized: now 1786262037597693 us; error 115 us; skew 500 ppm
I20260809 07:53:57.598158 31549 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:57 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262037, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:57.601439 31549 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:53:57.602941 31549 webserver.cc:533] Webserver started at http://127.26.37.61:43579/ using document root <none> and password file <none>
I20260809 07:53:57.603605 31549 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:57.603699 31549 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:57.603981 31549 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:57.606372 31549 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "de8b8e62a5234474993b7c1c26ab23f2"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:53:57.607158 31549 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "de8b8e62a5234474993b7c1c26ab23f2"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:53:57.612558 31549 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 07:53:57.615953 31568 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:57.617439 31549 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:53:57.617661 31549 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "de8b8e62a5234474993b7c1c26ab23f2"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:53:57.617861 31549 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:57.630753 31549 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:57.634728 31549 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:57.635030 31549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:57.655882 31549 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:39893
I20260809 07:53:57.655905 31635 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:39893 every 8 connection(s)
I20260809 07:53:57.657517 31549 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:53:57.661796 31637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:57.665624 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31549
I20260809 07:53:57.665831 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:53:57.668797 31637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:57 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262037, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:53:57.692473 31637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:57 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262037, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
I20260809 07:53:57.708974 31637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:53:57.710193 31569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
W20260809 07:53:57.711959 31637 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:57.761314 31637 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 1
W20260809 07:53:57.763599 31637 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:53:57.777690 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } attempt: 5
Aug 09 07:53:57 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262037, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:57.792630 31533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:53:57.794644 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:57.816926 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:33735
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:57.831856 31637 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 2
W20260809 07:53:57.834221 31637 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
I20260809 07:53:57.845893 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 1
W20260809 07:53:57.847944 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
I20260809 07:53:57.916226 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 2
W20260809 07:53:57.918393 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
I20260809 07:53:57.925441 31637 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 3
W20260809 07:53:57.927400 31637 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
I20260809 07:53:58.024689 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 3
W20260809 07:53:58.027040 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
I20260809 07:53:58.070641 31637 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 4
W20260809 07:53:58.073084 31637 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
W20260809 07:53:58.172749 31652 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:58.173055 31652 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:58.173223 31652 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:58.181097 31652 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:58.181228 31652 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:58.181380 31652 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:58.181475 31652 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:58.191705 31652 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:33735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:58.193517 31652 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:58.194905 31652 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:58.202977 31658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:58.203130 31657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:58.203590 31661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:58.204299 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 4
I20260809 07:53:58.204720 31652 server_base.cc:1061] running on GCE node
I20260809 07:53:58.205679 31652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260809 07:53:58.206300 31533 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111)
I20260809 07:53:58.207453 31652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:58.208660 31652 hybrid_clock.cc:648] HybridClock initialized: now 1786262038208535 us; error 91 us; skew 500 ppm
I20260809 07:53:58.208998 31652 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:58.212245 31652 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:53:58.213806 31652 webserver.cc:533] Webserver started at http://127.26.37.60:46365/ using document root <none> and password file <none>
I20260809 07:53:58.214570 31652 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:58.214710 31652 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:58.215061 31652 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:58.217465 31652 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "690c94c190be426bb6b79ee2f18b9166"
format_stamp: "Formatted at 2026-08-09 07:53:58 on dist-test-slave-bw88"
I20260809 07:53:58.218346 31652 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "690c94c190be426bb6b79ee2f18b9166"
format_stamp: "Formatted at 2026-08-09 07:53:58 on dist-test-slave-bw88"
I20260809 07:53:58.223706 31652 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:53:58.226990 31675 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:58.228319 31652 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:53:58.228529 31652 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "690c94c190be426bb6b79ee2f18b9166"
format_stamp: "Formatted at 2026-08-09 07:53:58 on dist-test-slave-bw88"
I20260809 07:53:58.228720 31652 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:58.247400 31652 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:58.251117 31652 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:58.251418 31652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:58.272769 31652 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:33735
I20260809 07:53:58.272826 31739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:33735 every 8 connection(s)
I20260809 07:53:58.274444 31652 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:53:58.278846 31741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:58.279708 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31652
I20260809 07:53:58.279938 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:53:58.286980 31741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:53:58.308713 31741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:58.322019 31741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262038, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:58.339962 31741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Bootstrap starting.
I20260809 07:53:58.343380 31741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:58.344514 31741 log.cc:826] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:58.346956 31741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: No bootstrap required, opened a new log
I20260809 07:53:58.352329 31741 raft_consensus.cc:359] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.352659 31741 raft_consensus.cc:385] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:53:58.352793 31741 raft_consensus.cc:740] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690c94c190be426bb6b79ee2f18b9166, State: Initialized, Role: FOLLOWER
I20260809 07:53:58.353438 31741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.354280 31752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.354499 31752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:58.355315 31741 sys_catalog.cc:565] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:53:58.365465 31771 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:58.365605 31771 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:58.362725 31652 master_runner.cc:187] Error getting master registration for 127.26.37.62:33383: OK, instance_id { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" instance_seqno: 1786262037064261 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:53:58.367878 31652 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:58.372321 31637 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 5
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262037, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:58.391453 31637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: Bootstrap starting.
I20260809 07:53:58.394886 31637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:58.395923 31637 log.cc:826] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:58.398543 31637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: No bootstrap required, opened a new log
I20260809 07:53:58.404253 31637 raft_consensus.cc:359] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.404469 31637 raft_consensus.cc:385] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:53:58.404627 31637 raft_consensus.cc:740] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8b8e62a5234474993b7c1c26ab23f2, State: Initialized, Role: FOLLOWER
I20260809 07:53:58.405224 31637 consensus_queue.cc:260] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.406164 31774 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.406424 31774 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:58.407286 31637 sys_catalog.cc:565] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:53:58.417385 31791 catalog_manager.cc:1594] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:58.417572 31791 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:58.414623 31549 master_runner.cc:187] Error getting master registration for 127.26.37.62:33383: OK, instance_id { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" instance_seqno: 1786262037064261 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:53:58.423794 31549 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:58.483752 31533 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } attempt: 5
Aug 09 07:53:58 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262037, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
W20260809 07:53:58.503664 31683 tablet.cc:2406] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:53:58.507236 31533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap starting.
I20260809 07:53:58.511854 31752 raft_consensus.cc:493] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:53:58.512159 31752 raft_consensus.cc:515] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.513741 31533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:58.513952 31752 leader_election.cc:290] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 943334b0c66e48a7b966b6623474b72f (127.26.37.62:33383), de8b8e62a5234474993b7c1c26ab23f2 (127.26.37.61:39893)
I20260809 07:53:58.515244 31533 log.cc:826] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:58.515138 31606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "690c94c190be426bb6b79ee2f18b9166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de8b8e62a5234474993b7c1c26ab23f2" is_pre_election: true
I20260809 07:53:58.515676 31606 raft_consensus.cc:2468] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 690c94c190be426bb6b79ee2f18b9166 in term 0.
I20260809 07:53:58.517325 31677 leader_election.cc:304] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 690c94c190be426bb6b79ee2f18b9166, de8b8e62a5234474993b7c1c26ab23f2; no voters:
I20260809 07:53:58.517895 31752 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:53:58.518049 31752 raft_consensus.cc:493] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:53:58.518146 31752 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:58.521783 31752 raft_consensus.cc:515] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.524164 31606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "690c94c190be426bb6b79ee2f18b9166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de8b8e62a5234474993b7c1c26ab23f2"
I20260809 07:53:58.524367 31606 raft_consensus.cc:3060] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:58.527940 31606 raft_consensus.cc:2468] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 690c94c190be426bb6b79ee2f18b9166 in term 1.
I20260809 07:53:58.530787 31677 leader_election.cc:304] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 690c94c190be426bb6b79ee2f18b9166, de8b8e62a5234474993b7c1c26ab23f2; no voters:
I20260809 07:53:58.534819 31752 leader_election.cc:290] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [CANDIDATE]: Term 1 election: Requested vote from peers 943334b0c66e48a7b966b6623474b72f (127.26.37.62:33383), de8b8e62a5234474993b7c1c26ab23f2 (127.26.37.61:39893)
I20260809 07:53:58.535876 31752 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:53:58.536495 31752 raft_consensus.cc:697] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 1 LEADER]: Becoming Leader. State: Replica: 690c94c190be426bb6b79ee2f18b9166, State: Running, Role: LEADER
I20260809 07:53:58.541383 31752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.543077 31533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: No bootstrap required, opened a new log
I20260809 07:53:58.559159 31795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 690c94c190be426bb6b79ee2f18b9166. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.559530 31795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:58.557726 31533 raft_consensus.cc:359] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.560318 31533 raft_consensus.cc:385] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:53:58.560494 31533 raft_consensus.cc:740] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943334b0c66e48a7b966b6623474b72f, State: Initialized, Role: FOLLOWER
I20260809 07:53:58.561371 31533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:53:58.565495 31797 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:53:58.568013 31797 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:53:58.568400 31798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.568681 31798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:58.570763 31499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "690c94c190be426bb6b79ee2f18b9166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943334b0c66e48a7b966b6623474b72f"
I20260809 07:53:58.571229 31499 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:58.570739 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "690c94c190be426bb6b79ee2f18b9166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943334b0c66e48a7b966b6623474b72f" is_pre_election: true
I20260809 07:53:58.576148 31499 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 690c94c190be426bb6b79ee2f18b9166 in term 1.
I20260809 07:53:58.570772 31533 sys_catalog.cc:565] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:58.580595 31606 raft_consensus.cc:1275] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 1 FOLLOWER]: Refusing update from remote peer 690c94c190be426bb6b79ee2f18b9166: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:53:58.581686 31795 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:53:58.588881 31499 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Refusing update from remote peer 690c94c190be426bb6b79ee2f18b9166: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:53:58.589855 31795 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:53:58.616758 31798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 690c94c190be426bb6b79ee2f18b9166. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.617087 31798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:58.617640 31800 mvcc.cc:204] Tried to move back new op lower bound from 7316529310013575168 to 7316529309874610176. Current Snapshot: MvccSnapshot[applied={T|T < 7316529310013575168}]
I20260809 07:53:58.620335 31774 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 690c94c190be426bb6b79ee2f18b9166. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.620589 31774 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:58.628328 31805 mvcc.cc:204] Tried to move back new op lower bound from 7316529310013575168 to 7316529309874610176. Current Snapshot: MvccSnapshot[applied={T|T < 7316529310013575168}]
I20260809 07:53:58.633282 31798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.633383 31774 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.633535 31798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:58.633636 31774 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:58.637080 31795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.637408 31795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:58.639142 31795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:53:58.639421 31795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:58.640457 31797 catalog_manager.cc:1383] Generated new cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:53:58.640587 31797 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:53:58.672402 31797 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:53:58.674152 31797 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:53:58.700263 31797 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Generated new TSK 0
I20260809 07:53:58.701445 31797 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:53:58.721557 31824 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:53:58.721676 31824 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: loading cluster ID for follower catalog manager: success
I20260809 07:53:58.725821 31824 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: acquiring CA information for follower catalog manager: success
I20260809 07:53:58.728379 31824 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:58.909453 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:59.322690 31831 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:59.323030 31831 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:59.323199 31831 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:59.331461 31831 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:59.331689 31831 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:53:59.342036 31831 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:59.343608 31831 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:59.344938 31831 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:53:59.354890 31839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:59.354974 31838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:59.355412 31843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:59.356467 31831 server_base.cc:1061] running on GCE node
I20260809 07:53:59.357105 31831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:59.358282 31831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:59.359524 31831 hybrid_clock.cc:648] HybridClock initialized: now 1786262039359365 us; error 127 us; skew 500 ppm
I20260809 07:53:59.359824 31831 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:53:59 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262039, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:59.363193 31831 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:53:59.364766 31831 webserver.cc:533] Webserver started at http://127.26.37.1:34897/ using document root <none> and password file <none>
I20260809 07:53:59.365548 31831 fs_manager.cc:362] Metadata directory not provided
I20260809 07:53:59.365684 31831 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:59.366125 31831 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:59.368563 31831 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:53:59.369421 31831 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:53:59.374791 31831 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 07:53:59.378093 31851 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:59.379372 31831 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:53:59.379558 31831 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:53:59.379746 31831 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:59.398221 31831 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:53:59.402127 31831 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:59.402441 31831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:59.403393 31831 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:53:59.404968 31831 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:53:59.405045 31831 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:59.405205 31831 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:53:59.405267 31831 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:59.419948 31791 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:53:59.420085 31791 catalog_manager.cc:1588] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: loading cluster ID for follower catalog manager: success
I20260809 07:53:59.423615 31791 catalog_manager.cc:1610] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: acquiring CA information for follower catalog manager: success
I20260809 07:53:59.425495 31791 catalog_manager.cc:1638] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:53:59.444232 31831 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44721
I20260809 07:53:59.444322 31991 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44721 every 8 connection(s)
I20260809 07:53:59.445776 31831 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:53:59.452517 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 31831
I20260809 07:53:59.452767 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:53:59 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262039, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:53:59 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262039, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:53:59 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262039, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:53:59.489866 31992 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:53:59.489959 31993 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33735
I20260809 07:53:59.490254 31993 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:59.490262 31992 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:59.490319 31994 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39893
I20260809 07:53:59.490494 31994 heartbeater.cc:461] Registering TS with master...
I20260809 07:53:59.491366 31993 heartbeater.cc:507] Master 127.26.37.60:33735 requested a full tablet report, sending...
I20260809 07:53:59.491400 31994 heartbeater.cc:507] Master 127.26.37.61:39893 requested a full tablet report, sending...
I20260809 07:53:59.492650 31992 heartbeater.cc:507] Master 127.26.37.62:33383 requested a full tablet report, sending...
I20260809 07:53:59.494011 31592 ts_manager.cc:194] Registered new tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:44721)
I20260809 07:53:59.494622 31485 ts_manager.cc:194] Registered new tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:44721)
I20260809 07:53:59.496577 31695 ts_manager.cc:194] Registered new tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:44721)
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
I20260809 07:53:59.498193 31695 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:41743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:53:59.605593 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:59.972431 32011 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:59.972777 32011 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:59.972919 32011 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:59.981101 32011 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:59.981380 32011 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:53:59.992478 32011 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:53:59.994117 32011 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:59.995463 32011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:00.005923 32019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:00.006054 32021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:00.006816 32024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:00.007680 32011 server_base.cc:1061] running on GCE node
I20260809 07:54:00.008469 32011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:00.009730 32011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:00.011003 32011 hybrid_clock.cc:648] HybridClock initialized: now 1786262040010847 us; error 125 us; skew 500 ppm
I20260809 07:54:00.011344 32011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:00 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262040, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:00.014719 32011 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:54:00.016568 32011 webserver.cc:533] Webserver started at http://127.26.37.2:44929/ using document root <none> and password file <none>
I20260809 07:54:00.017683 32011 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:00.017831 32011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:00.018270 32011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:00.021554 32011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "f710f7c65ed84ff5a00862072d6caf2b"
format_stamp: "Formatted at 2026-08-09 07:54:00 on dist-test-slave-bw88"
I20260809 07:54:00.022382 32011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "f710f7c65ed84ff5a00862072d6caf2b"
format_stamp: "Formatted at 2026-08-09 07:54:00 on dist-test-slave-bw88"
I20260809 07:54:00.029027 32011 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 07:54:00.032632 32031 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:00.034188 32011 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:00.034370 32011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "f710f7c65ed84ff5a00862072d6caf2b"
format_stamp: "Formatted at 2026-08-09 07:54:00 on dist-test-slave-bw88"
I20260809 07:54:00.034584 32011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:00.051673 32011 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:00.058697 32011 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:00.059005 32011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:00.059902 32011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:00.061580 32011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:00.061685 32011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:00.061834 32011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:00.061908 32011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:00.099918 32011 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:38673
I20260809 07:54:00.099972 32173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:38673 every 8 connection(s)
I20260809 07:54:00.101522 32011 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:54:00.109917 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32011
I20260809 07:54:00.110172 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:54:00 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262040, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:54:00 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262040, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:54:00 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262040, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:54:00.130638 32176 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39893
I20260809 07:54:00.131089 32176 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:00.131654 32175 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33735
I20260809 07:54:00.131808 32175 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:00.132128 32176 heartbeater.cc:507] Master 127.26.37.61:39893 requested a full tablet report, sending...
I20260809 07:54:00.132366 32175 heartbeater.cc:507] Master 127.26.37.60:33735 requested a full tablet report, sending...
I20260809 07:54:00.134240 31695 ts_manager.cc:194] Registered new tserver with Master: f710f7c65ed84ff5a00862072d6caf2b (127.26.37.2:38673)
I20260809 07:54:00.134377 31592 ts_manager.cc:194] Registered new tserver with Master: f710f7c65ed84ff5a00862072d6caf2b (127.26.37.2:38673)
I20260809 07:54:00.135310 31695 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:56105
I20260809 07:54:00.137513 32174 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:00.137642 32174 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:00.137931 32174 heartbeater.cc:507] Master 127.26.37.62:33383 requested a full tablet report, sending...
I20260809 07:54:00.138877 31485 ts_manager.cc:194] Registered new tserver with Master: f710f7c65ed84ff5a00862072d6caf2b (127.26.37.2:38673)
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:54:00.253794 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:54:00.500593 31993 heartbeater.cc:499] Master 127.26.37.60:33735 was elected leader, sending a full tablet report...
W20260809 07:54:00.616183 32188 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:00.616503 32188 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:00.616595 32188 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:00.625056 32188 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:00.625242 32188 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:54:00.636299 32188 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:00.638131 32188 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:00.639444 32188 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:00.650388 32195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:00.650430 32194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:00.650676 32198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:00.651185 32188 server_base.cc:1061] running on GCE node
I20260809 07:54:00.651842 32188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:00.652917 32188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:00.654162 32188 hybrid_clock.cc:648] HybridClock initialized: now 1786262040654000 us; error 131 us; skew 500 ppm
I20260809 07:54:00.654521 32188 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:00 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262040, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:00.658478 32188 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:54:00.660326 32188 webserver.cc:533] Webserver started at http://127.26.37.3:36779/ using document root <none> and password file <none>
I20260809 07:54:00.661146 32188 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:00.661298 32188 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:00.661671 32188 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:00.664621 32188 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "df79b59db14d45f1a6997dbb849af08e"
format_stamp: "Formatted at 2026-08-09 07:54:00 on dist-test-slave-bw88"
I20260809 07:54:00.665607 32188 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "df79b59db14d45f1a6997dbb849af08e"
format_stamp: "Formatted at 2026-08-09 07:54:00 on dist-test-slave-bw88"
I20260809 07:54:00.671383 32188 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 07:54:00.674638 32205 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:00.675935 32188 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:54:00.676123 32188 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "df79b59db14d45f1a6997dbb849af08e"
format_stamp: "Formatted at 2026-08-09 07:54:00 on dist-test-slave-bw88"
I20260809 07:54:00.676338 32188 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:00.695701 32188 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:00.699647 32188 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:00.699888 32188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:00.700762 32188 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:00.702477 32188 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:00.702570 32188 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:00.702656 32188 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:00.702701 32188 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:00.740185 32188 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:35773
I20260809 07:54:00.740272 32349 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:35773 every 8 connection(s)
I20260809 07:54:00.741609 32188 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:54:00.750126 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32188
I20260809 07:54:00.750362 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:54:00 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262040, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:54:00 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262040, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:54:00 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262040, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:54:00.770251 32350 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:00.770632 32350 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:00.771802 32350 heartbeater.cc:507] Master 127.26.37.62:33383 requested a full tablet report, sending...
I20260809 07:54:00.772794 32352 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33735
I20260809 07:54:00.772948 32352 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:00.773528 32352 heartbeater.cc:507] Master 127.26.37.60:33735 requested a full tablet report, sending...
I20260809 07:54:00.774005 31485 ts_manager.cc:194] Registered new tserver with Master: df79b59db14d45f1a6997dbb849af08e (127.26.37.3:35773)
I20260809 07:54:00.774947 31694 ts_manager.cc:194] Registered new tserver with Master: df79b59db14d45f1a6997dbb849af08e (127.26.37.3:35773)
I20260809 07:54:00.776201 31694 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:52987
I20260809 07:54:00.776809 32353 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39893
I20260809 07:54:00.776986 32353 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:00.777611 32353 heartbeater.cc:507] Master 127.26.37.61:39893 requested a full tablet report, sending...
I20260809 07:54:00.778767 31592 ts_manager.cc:194] Registered new tserver with Master: df79b59db14d45f1a6997dbb849af08e (127.26.37.3:35773)
I20260809 07:54:00.792086 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:54:00.799502 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:54:00.799687 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31444
I20260809 07:54:00.800566 31528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:00.908720 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31444
I20260809 07:54:00.940551 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33383
--webserver_interface=127.26.37.62
--webserver_port=46753
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:01.006073 31679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:33383: connect: Connection refused (error 111)
W20260809 07:54:01.008235 31679 consensus_peers.cc:597] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 -> Peer 943334b0c66e48a7b966b6623474b72f (127.26.37.62:33383): Couldn't send request to peer 943334b0c66e48a7b966b6623474b72f. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:33383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:54:01.137960 32175 heartbeater.cc:499] Master 127.26.37.60:33735 was elected leader, sending a full tablet report...
W20260809 07:54:01.141098 32174 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:33383 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:33383: connect: Connection refused (error 111)
W20260809 07:54:01.306782 32364 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:01.307127 32364 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:01.307297 32364 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:01.315531 32364 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:01.315620 32364 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:01.315713 32364 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:01.315799 32364 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:01.326007 32364 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=46753
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:01.327631 32364 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:01.328994 32364 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:01.337244 32374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:01.337316 32375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:01.338179 32377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:01.338541 32364 server_base.cc:1061] running on GCE node
I20260809 07:54:01.339337 32364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:01.340884 32364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:01.344183 32364 hybrid_clock.cc:648] HybridClock initialized: now 1786262041343958 us; error 195 us; skew 500 ppm
I20260809 07:54:01.344494 32364 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:01 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262041, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:01.347820 32364 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:54:01.349539 32364 webserver.cc:533] Webserver started at http://127.26.37.62:46753/ using document root <none> and password file <none>
I20260809 07:54:01.350244 32364 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:01.350354 32364 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:01.355911 32364 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:54:01.358969 32386 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:01.360280 32364 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:01.360419 32364 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "943334b0c66e48a7b966b6623474b72f"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:54:01.361092 32364 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:01.389791 32364 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:01.390753 31795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:01.391048 31795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:01.394948 32364 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:01.395179 32364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:01.417650 32460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33383 every 8 connection(s)
I20260809 07:54:01.417655 32364 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33383
I20260809 07:54:01.419317 32364 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:01.424057 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32364
I20260809 07:54:01.424494 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31831
I20260809 07:54:01.428016 32461 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:01.431851 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap starting.
Aug 09 07:54:01 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:54:01.444412 31987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:01.454088 32461 log.cc:826] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:01.463796 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:01.464365 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap complete.
I20260809 07:54:01.471701 32461 raft_consensus.cc:359] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:01.472419 32461 raft_consensus.cc:740] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943334b0c66e48a7b966b6623474b72f, State: Initialized, Role: FOLLOWER
I20260809 07:54:01.473178 32461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:01.474280 32468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:01.474633 32468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:01.475445 32461 sys_catalog.cc:565] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:01.486816 32480 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:54:01.486999 32480 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: loading cluster ID for follower catalog manager: success
I20260809 07:54:01.492336 32480 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: acquiring CA information for follower catalog manager: success
I20260809 07:54:01.495733 32480 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:01.564420 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31831
W20260809 07:54:01.590914 31569 connection.cc:573] server connection from 127.26.37.1:53345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:54:01.591401 31679 connection.cc:573] server connection from 127.26.37.1:41743 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:01.592139 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44721
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--webserver_port=34897
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:01.592316 32484 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:01.563174 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:59865 (local address 127.26.37.62:33383)
0809 07:54:01.564267 (+ 1093us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:01.564273 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:01.564295 (+ 22us) server_negotiation.cc:408] Connection header received
0809 07:54:01.564368 (+ 73us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:01.564374 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:01.564447 (+ 73us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:01.564559 (+ 112us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0809 07:54:01.590935 (+ 26376us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:59865: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":966}
I20260809 07:54:01.779111 32352 heartbeater.cc:499] Master 127.26.37.60:33735 was elected leader, sending a full tablet report...
I20260809 07:54:01.789745 32415 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "df79b59db14d45f1a6997dbb849af08e" instance_seqno: 1786262040730891) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:50217; Asking this server to re-register.
I20260809 07:54:01.790652 32350 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:01.792569 32415 ts_manager.cc:194] Registered new tserver with Master: df79b59db14d45f1a6997dbb849af08e (127.26.37.3:35773)
I20260809 07:54:01.813223 32468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 690c94c190be426bb6b79ee2f18b9166. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:01.813581 32468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:54:01.981014 32490 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:01.981451 32490 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:01.981631 32490 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:01.989746 32490 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:01.989967 32490 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:02.000432 32490 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34897
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:02.002197 32490 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:02.003521 32490 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:02.013841 32500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:02.013819 32501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:02.014067 32503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:02.015144 32490 server_base.cc:1061] running on GCE node
I20260809 07:54:02.015699 32490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:02.016767 32490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:02.018002 32490 hybrid_clock.cc:648] HybridClock initialized: now 1786262042017854 us; error 117 us; skew 500 ppm
I20260809 07:54:02.018338 32490 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:02 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262042, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:02.021690 32490 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:54:02.023307 32490 webserver.cc:533] Webserver started at http://127.26.37.1:34897/ using document root <none> and password file <none>
I20260809 07:54:02.023989 32490 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:02.024097 32490 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:02.029264 32490 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:02.032028 32511 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:02.033612 32490 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:54:02.033788 32490 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:54:02.034511 32490 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:02.056463 32490 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:02.060426 32490 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:02.060735 32490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:02.061769 32490 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:02.063421 32490 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:02.063560 32490 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:02.063695 32490 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:02.063784 32490 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:02.107402 32490 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44721
I20260809 07:54:02.107473 32650 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44721 every 8 connection(s)
I20260809 07:54:02.109365 32490 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:02.111089 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32490
Aug 09 07:54:02 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:54:02 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262042, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:54:02 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262042, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:54:02.170732 32652 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:02.171166 32652 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:02.172329 32652 heartbeater.cc:507] Master 127.26.37.62:33383 requested a full tablet report, sending...
I20260809 07:54:02.173475 32654 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39893
I20260809 07:54:02.173625 32654 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:02.174190 32654 heartbeater.cc:507] Master 127.26.37.61:39893 requested a full tablet report, sending...
I20260809 07:54:02.175597 32415 ts_manager.cc:194] Registered new tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:44721)
I20260809 07:54:02.175634 31592 ts_manager.cc:194] Re-registered known tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:44721)
Aug 09 07:54:02 dist-test-slave-bw88 krb5kdc[31063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262042, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:54:02.178872 32174 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:02.180205 32415 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "f710f7c65ed84ff5a00862072d6caf2b" instance_seqno: 1786262040091202) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:40725; Asking this server to re-register.
I20260809 07:54:02.180775 32174 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:02.181984 32415 ts_manager.cc:194] Registered new tserver with Master: f710f7c65ed84ff5a00862072d6caf2b (127.26.37.2:38673)
I20260809 07:54:02.187103 32653 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33735
I20260809 07:54:02.187249 32653 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:02.187702 32653 heartbeater.cc:507] Master 127.26.37.60:33735 requested a full tablet report, sending...
I20260809 07:54:02.188995 31694 ts_manager.cc:194] Re-registered known tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:44721)
I20260809 07:54:02.190130 31694 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:43191
I20260809 07:54:02.293290 31795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "690c94c190be426bb6b79ee2f18b9166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:02.293565 31795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:02.308513 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32490
W20260809 07:54:02.310428 32673 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:02.305733 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46472 (local address 127.26.37.1:44721)
0809 07:54:02.306584 (+ 851us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:02.306591 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:02.306634 (+ 43us) server_negotiation.cc:408] Connection header received
0809 07:54:02.306692 (+ 58us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:02.306698 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:02.306922 (+ 224us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:02.307110 (+ 188us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:02.308252 (+ 1142us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46472: BlockingRecv error: recv got EOF from 127.0.0.1:46472 (error 108)
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":471,"threads_started":1}
W20260809 07:54:02.311362 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32490: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:44721: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:54:02.311524 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32490
I20260809 07:54:02.338548 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44721
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--webserver_port=34897
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:02.710228 32676 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:02.710652 32676 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:02.710855 32676 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:54:02.710976 32676 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:02.720281 32676 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:02.720580 32676 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:02.731798 32676 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34897
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:02.733641 32676 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:02.735097 32676 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:02.745076 32684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:02.745116 32682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:02.746519 32687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:02.747524 32676 server_base.cc:1061] running on GCE node
I20260809 07:54:02.748057 32676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:02.749109 32676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:02.750423 32676 hybrid_clock.cc:648] HybridClock initialized: now 1786262042750301 us; error 78 us; skew 500 ppm
I20260809 07:54:02.750741 32676 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:02 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262042, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:02.754422 32676 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:54:02.756230 32676 webserver.cc:533] Webserver started at http://127.26.37.1:34897/ using document root <none> and password file <none>
I20260809 07:54:02.756994 32676 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:02.757081 32676 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:02.763545 32676 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:54:02.766500 32696 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:02.767910 32676 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:02.768105 32676 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:54:02.768805 32676 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:02.798543 32676 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:02.802359 32676 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:02.802708 32676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:02.803735 32676 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:02.805457 32676 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:02.805538 32676 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:02.805665 32676 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:02.805722 32676 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:54:02.806136 32676 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:54:02.806232 32676 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:54:02.806344 32676 fault_injection.cc:59] injecting fault for kudu (pid 32676): FLAGS_fault_before_start (process will exit)
I20260809 07:54:02.834308 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44721
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--webserver_port=34897
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:03.209087 32707 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:03.209506 32707 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:03.209733 32707 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:03.218381 32707 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:03.218633 32707 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:03.230156 32707 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34145,127.26.37.21:45303,127.26.37.22:39437,127.26.37.23:45245,127.26.37.24:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34897
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:03.231992 32707 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:03.233594 32707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:03.246922 32712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:03.247010 32713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:03.247896 32715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:03.248860 32707 server_base.cc:1061] running on GCE node
I20260809 07:54:03.249581 32707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:03.250765 32707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:03.251983 32707 hybrid_clock.cc:648] HybridClock initialized: now 1786262043251848 us; error 104 us; skew 500 ppm
I20260809 07:54:03.252322 32707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:03 dist-test-slave-bw88 krb5kdc[31063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262043, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:03.256006 32707 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:54:03.257783 32707 webserver.cc:533] Webserver started at http://127.26.37.1:34897/ using document root <none> and password file <none>
I20260809 07:54:03.258646 32707 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:03.258800 32707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:03.266501 32707 fs_manager.cc:714] Time spent opening directory manager: real 0.006s user 0.004s sys 0.003s
I20260809 07:54:03.269671 32722 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:03.271265 32707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:03.271471 32707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:54:03.272270 32707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:03.297019 32707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:03.300992 32707 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:03.301344 32707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:03.302402 32707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:03.304107 32707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:03.304241 32707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:03.304378 32707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:03.304476 32707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:03.342425 32707 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44721
I20260809 07:54:03.342518 398 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44721 every 8 connection(s)
I20260809 07:54:03.343994 32707 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:03.351277 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 32707
I20260809 07:54:03.351773 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32707
W20260809 07:54:03.379604 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32707: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:44721: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:54:03.379844 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32707
W20260809 07:54:03.406251 403 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:03.347376 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:42051 (local address 127.26.37.62:33383)
0809 07:54:03.393829 (+ 46453us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:03.393838 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:03.393862 (+ 24us) server_negotiation.cc:408] Connection header received
0809 07:54:03.393936 (+ 74us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:03.393945 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:03.394044 (+ 99us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:03.394184 (+ 140us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:03.405831 (+ 11647us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:42051: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":46286,"thread_start_us":46150,"threads_started":1}
I20260809 07:54:03.406219 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32011
W20260809 07:54:03.406842 412 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:03.372948 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:44851 (local address 127.26.37.60:33735)
0809 07:54:03.373720 (+ 772us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:03.373729 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:03.378457 (+ 4728us) server_negotiation.cc:408] Connection header received
0809 07:54:03.378596 (+ 139us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:03.378603 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:03.378714 (+ 111us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:03.378865 (+ 151us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:03.405871 (+ 27006us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:44851: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":254,"threads_started":1}
W20260809 07:54:03.406841 404 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:03.347430 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:37223 (local address 127.26.37.61:39893)
0809 07:54:03.378896 (+ 31466us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:03.378905 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:03.378926 (+ 21us) server_negotiation.cc:408] Connection header received
0809 07:54:03.378991 (+ 65us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:03.378997 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:03.379076 (+ 79us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:03.379196 (+ 120us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:03.405930 (+ 26734us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:37223: BlockingRecv error: recv got EOF from 127.26.37.1:37223 (error 108)
Metrics: {"server-negotiator.queue_time_us":31339,"thread_start_us":31935,"threads_started":1}
W20260809 07:54:03.409192 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32011: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:38673: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:54:03.409384 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32011
I20260809 07:54:03.434553 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32188
W20260809 07:54:03.437880 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32188: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:35773: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:54:03.438077 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32188
I20260809 07:54:03.463186 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32364
W20260809 07:54:03.465449 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 32364: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:33383: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:54:03.465524 403 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:03.464058 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45790 (local address 127.26.37.62:33383)
0809 07:54:03.464230 (+ 172us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:03.464234 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:03.464333 (+ 99us) server_negotiation.cc:408] Connection header received
0809 07:54:03.464487 (+ 154us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:03.464492 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:03.464558 (+ 66us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:03.464643 (+ 85us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:03.465294 (+ 651us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45790: BlockingRecv error: recv got EOF from 127.0.0.1:45790 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 07:54:03.465606 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 32364
I20260809 07:54:03.493930 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31549
I20260809 07:54:03.495148 31631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:03.607262 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31549
I20260809 07:54:03.635251 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 31652
I20260809 07:54:03.636564 31735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:03.749517 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 31652
Shutting down hive metastore.
2026-08-09T07:54:04Z chronyd exiting
2026-08-09T07:54:04Z chronyd exiting
2026-08-09T07:54:04Z chronyd exiting
2026-08-09T07:54:04Z chronyd exiting
2026-08-09T07:54:04Z chronyd exiting
Aug 09 07:54:04 dist-test-slave-bw88 krb5kdc[425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:54:04 dist-test-slave-bw88 krb5kdc[425](info): set up 2 sockets
Aug 09 07:54:04 dist-test-slave-bw88 krb5kdc[425](info): commencing operation
krb5kdc: starting...
W20260809 07:54:06.392189 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.179s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:54:06 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:54:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:06Z Disabled control of system clock
2026-08-09T07:54:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:06Z Disabled control of system clock
2026-08-09T07:54:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:06Z Disabled control of system clock
2026-08-09T07:54:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:06Z Disabled control of system clock
2026-08-09T07:54:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:06Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "hive/127.0.0.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:54:07: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:54:10 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262050, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:54:14 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262054, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:54:14 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262054, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.62@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:54:16.633141 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33383
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:16.988469 677 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:16.988830 677 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:16.988997 677 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:16.997634 677 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:16.997753 677 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:16.997846 677 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:16.997931 677 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:17.008589 677 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:17.010314 677 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:17.011811 677 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:17.020843 682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:17.021059 684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:17.021991 689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:17.022792 677 server_base.cc:1061] running on GCE node
I20260809 07:54:17.023550 677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:17.025102 677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:17.026336 677 hybrid_clock.cc:648] HybridClock initialized: now 1786262057026197 us; error 108 us; skew 500 ppm
I20260809 07:54:17.026718 677 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:17 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262057, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:17.030231 677 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:54:17.031950 677 webserver.cc:533] Webserver started at http://127.26.37.62:38079/ using document root <none> and password file <none>
I20260809 07:54:17.032703 677 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:17.032868 677 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:17.038265 677 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:54:17.041082 698 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:17.042443 677 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:17.042658 677 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "943334b0c66e48a7b966b6623474b72f"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:54:17.043372 677 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:17.075778 677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:17.079820 677 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:17.080129 677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:17.105696 677 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33383
I20260809 07:54:17.105790 771 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33383 every 8 connection(s)
I20260809 07:54:17.107362 677 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:17.114895 772 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:17.116269 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 677
I20260809 07:54:17.116478 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:54:17.119462 772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap starting.
I20260809 07:54:17.140029 772 log.cc:826] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:54:17.149582 772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:17.150224 772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap complete.
I20260809 07:54:17.161772 772 raft_consensus.cc:359] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:17.162883 772 raft_consensus.cc:740] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943334b0c66e48a7b966b6623474b72f, State: Initialized, Role: FOLLOWER
I20260809 07:54:17.163668 772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:17.164623 778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:17.164971 778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:17.165781 772 sys_catalog.cc:565] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:17.183156 793 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:54:17.184016 793 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: loading cluster ID for follower catalog manager: success
I20260809 07:54:17.189052 793 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: acquiring CA information for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:54:17.191939 793 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:54:17 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262057, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:54:17.207216 677 master_runner.cc:187] Error getting master registration for 127.26.37.61:39893: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39893: connect: Connection refused (error 111),
I20260809 07:54:17.210906 677 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.61@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:54:17.272198 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39893
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:17.661335 800 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:17.661746 800 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:17.661907 800 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:17.670626 800 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:17.670723 800 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:17.670842 800 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:17.670924 800 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:17.682082 800 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:17.683748 800 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:17.685178 800 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:17.693994 809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:17.694067 806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:17.694067 807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:17.694965 800 server_base.cc:1061] running on GCE node
I20260809 07:54:17.695686 800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:17.697293 800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:17.698581 800 hybrid_clock.cc:648] HybridClock initialized: now 1786262057698437 us; error 111 us; skew 500 ppm
I20260809 07:54:17.698948 800 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:17 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262057, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:17.702489 800 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:54:17.704216 800 webserver.cc:533] Webserver started at http://127.26.37.61:42519/ using document root <none> and password file <none>
I20260809 07:54:17.705111 800 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:17.705273 800 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:17.710999 800 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:54:17.713809 817 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:17.715170 800 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:17.715363 800 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "de8b8e62a5234474993b7c1c26ab23f2"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:54:17.716068 800 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:17.741278 800 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:17.745440 800 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:17.745747 800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:17.767199 800 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:39893
I20260809 07:54:17.767220 884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:39893 every 8 connection(s)
I20260809 07:54:17.768963 800 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:54:17.776054 885 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:17.778857 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 800
I20260809 07:54:17.779090 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:54:17.779647 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: Bootstrap starting.
I20260809 07:54:17.797856 885 log.cc:826] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:17.810000 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:17.810730 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: Bootstrap complete.
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:54:17.820561 885 raft_consensus.cc:359] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:17.821493 885 raft_consensus.cc:740] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8b8e62a5234474993b7c1c26ab23f2, State: Initialized, Role: FOLLOWER
I20260809 07:54:17.822367 885 consensus_queue.cc:260] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:17.823275 890 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:17.823606 890 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:17.824262 885 sys_catalog.cc:565] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:17.834676 905 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:54:17.834841 905 catalog_manager.cc:1588] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: loading cluster ID for follower catalog manager: success
I20260809 07:54:17.839214 905 catalog_manager.cc:1610] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: acquiring CA information for follower catalog manager: success
I20260809 07:54:17.841921 905 catalog_manager.cc:1638] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:54:17 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262057, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.60@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:54:17.894531 800 master_runner.cc:187] Error getting master registration for 127.26.37.60:33735: Network error: Client connection negotiation failed: client connection to 127.26.37.60:33735: connect: Connection refused (error 111),
I20260809 07:54:17.900403 800 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:54:17.936939 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:33735
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:18.299592 915 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:18.299962 915 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:18.300108 915 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:18.308180 915 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:18.308327 915 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:18.308434 915 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:18.308537 915 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:18.319775 915 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:33735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:18.321702 915 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:18.323134 915 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:18.332152 922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:18.333310 923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:18.334044 927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:18.335076 915 server_base.cc:1061] running on GCE node
I20260809 07:54:18.335888 915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:18.337507 915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:18.338737 915 hybrid_clock.cc:648] HybridClock initialized: now 1786262058338597 us; error 92 us; skew 500 ppm
I20260809 07:54:18.339114 915 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:18 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262058, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:18.342826 915 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:54:18.344378 915 webserver.cc:533] Webserver started at http://127.26.37.60:44111/ using document root <none> and password file <none>
I20260809 07:54:18.345057 915 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:18.345166 915 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:18.350806 915 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 07:54:18.353742 938 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:18.355108 915 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:18.355295 915 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "690c94c190be426bb6b79ee2f18b9166"
format_stamp: "Formatted at 2026-08-09 07:53:58 on dist-test-slave-bw88"
I20260809 07:54:18.356020 915 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:18.370492 915 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:18.374485 915 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:18.374755 915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:18.395773 998 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:33735 every 8 connection(s)
I20260809 07:54:18.395817 915 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:33735
I20260809 07:54:18.397536 915 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:54:18.400044 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 915
I20260809 07:54:18.400257 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:54:18.406162 999 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:18.412735 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Bootstrap starting.
Aug 09 07:54:18 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:54:18 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:54:18.432932 999 log.cc:826] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:18.442358 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:18.442922 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: Bootstrap complete.
Aug 09 07:54:18 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:54:18.449785 999 raft_consensus.cc:359] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:18.450858 999 raft_consensus.cc:740] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690c94c190be426bb6b79ee2f18b9166, State: Initialized, Role: FOLLOWER
I20260809 07:54:18.451656 999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:18.452556 1007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:18.452893 1007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:18.453972 999 sys_catalog.cc:565] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:18.467898 1022 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:54:18.468007 1022 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: loading cluster ID for follower catalog manager: success
I20260809 07:54:18.471268 1022 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: acquiring CA information for follower catalog manager: success
I20260809 07:54:18.474035 1022 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:54:18 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262058, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:54:18.502684 915 master_runner.cc:264] No leader master found from master 690c94c190be426bb6b79ee2f18b9166
I20260809 07:54:18.502844 915 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:54:18.505160 1027 raft_consensus.cc:493] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:54:18.505455 1027 raft_consensus.cc:515] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:18.507349 1027 leader_election.cc:290] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de8b8e62a5234474993b7c1c26ab23f2 (127.26.37.61:39893), 690c94c190be426bb6b79ee2f18b9166 (127.26.37.60:33735)
I20260809 07:54:18.515545 856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "943334b0c66e48a7b966b6623474b72f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "de8b8e62a5234474993b7c1c26ab23f2" is_pre_election: true
I20260809 07:54:18.516131 856 raft_consensus.cc:2468] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 943334b0c66e48a7b966b6623474b72f in term 1.
I20260809 07:54:18.516747 701 leader_election.cc:304] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 943334b0c66e48a7b966b6623474b72f, de8b8e62a5234474993b7c1c26ab23f2; no voters:
I20260809 07:54:18.517215 1027 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:54:18.517416 1027 raft_consensus.cc:493] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:54:18.517539 1027 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:54:18.517922 970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "943334b0c66e48a7b966b6623474b72f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "690c94c190be426bb6b79ee2f18b9166" is_pre_election: true
I20260809 07:54:18.518653 970 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 943334b0c66e48a7b966b6623474b72f in term 1.
I20260809 07:54:18.523250 1027 raft_consensus.cc:515] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:18.523972 1027 leader_election.cc:290] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [CANDIDATE]: Term 2 election: Requested vote from peers de8b8e62a5234474993b7c1c26ab23f2 (127.26.37.61:39893), 690c94c190be426bb6b79ee2f18b9166 (127.26.37.60:33735)
I20260809 07:54:18.525192 856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "943334b0c66e48a7b966b6623474b72f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "de8b8e62a5234474993b7c1c26ab23f2"
I20260809 07:54:18.525398 970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "943334b0c66e48a7b966b6623474b72f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "690c94c190be426bb6b79ee2f18b9166"
I20260809 07:54:18.525426 856 raft_consensus.cc:3060] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:54:18.525643 970 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:54:18.530990 856 raft_consensus.cc:2468] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 943334b0c66e48a7b966b6623474b72f in term 2.
I20260809 07:54:18.531056 970 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 943334b0c66e48a7b966b6623474b72f in term 2.
I20260809 07:54:18.531479 701 leader_election.cc:304] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 943334b0c66e48a7b966b6623474b72f, de8b8e62a5234474993b7c1c26ab23f2; no voters:
I20260809 07:54:18.531829 1027 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:54:18.532326 1027 raft_consensus.cc:697] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 2 LEADER]: Becoming Leader. State: Replica: 943334b0c66e48a7b966b6623474b72f, State: Running, Role: LEADER
I20260809 07:54:18.532905 1027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:18.538066 1031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 943334b0c66e48a7b966b6623474b72f. Latest consensus state: current_term: 2 leader_uuid: "943334b0c66e48a7b966b6623474b72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:18.538333 1031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:18.539075 1035 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:54:18.541018 1035 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:54:18.541911 1035 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:54:18.542026 1035 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:54:18.543154 1035 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:54:18.543931 1035 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Loaded TSK: 0
I20260809 07:54:18.544857 1035 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:54:18.700475 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:54:18.995345 970 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 2 FOLLOWER]: Refusing update from remote peer 943334b0c66e48a7b966b6623474b72f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:54:18.996459 1031 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [LEADER]: Connected to new peer: Peer: permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:54:19.000358 1045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 943334b0c66e48a7b966b6623474b72f. Latest consensus state: current_term: 2 leader_uuid: "943334b0c66e48a7b966b6623474b72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:19.000638 1045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:19.001856 1027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "943334b0c66e48a7b966b6623474b72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:19.002161 1027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:19.003881 856 raft_consensus.cc:1275] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 2 FOLLOWER]: Refusing update from remote peer 943334b0c66e48a7b966b6623474b72f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 07:54:19.003916 1045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "943334b0c66e48a7b966b6623474b72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:19.004117 1045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:19.004765 1052 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [LEADER]: Connected to new peer: Peer: permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:54:19.008960 1056 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "943334b0c66e48a7b966b6623474b72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:19.009431 1056 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:19.009820 1053 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 943334b0c66e48a7b966b6623474b72f. Latest consensus state: current_term: 2 leader_uuid: "943334b0c66e48a7b966b6623474b72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:19.010049 1053 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:54:19.072265 1044 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:19.072660 1044 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:19.072816 1044 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:19.081023 1044 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:19.081270 1044 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:19.093892 1044 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:19.095956 1044 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:19.097450 1044 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:19.111446 1063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:19.111529 1062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:19.112625 1065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:19.113706 1044 server_base.cc:1061] running on GCE node
I20260809 07:54:19.114395 1044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:19.115906 1044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:19.117138 1044 hybrid_clock.cc:648] HybridClock initialized: now 1786262059116986 us; error 116 us; skew 500 ppm
I20260809 07:54:19.117569 1044 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:19 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:19.122126 1044 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:54:19.123884 1044 webserver.cc:533] Webserver started at http://127.26.37.1:44831/ using document root <none> and password file <none>
I20260809 07:54:19.124688 1044 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:19.124830 1044 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:19.130606 1044 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:54:19.133487 1073 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:19.134984 1044 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:19.135172 1044 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:54:19.135932 1044 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:19.151587 1044 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:19.155833 1044 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:19.156145 1044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:19.157163 1044 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:19.158838 1044 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:19.158977 1044 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:19.159111 1044 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:19.159198 1044 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:19.194502 1044 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:38797
I20260809 07:54:19.194594 1225 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:38797 every 8 connection(s)
I20260809 07:54:19.196183 1044 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:19.206543 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1044
I20260809 07:54:19.206833 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:54:19 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:54:19 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:54:19 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:54:19.251778 1226 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:19.251868 1227 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33735
I20260809 07:54:19.252240 1226 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:19.252240 1227 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:19.252290 1228 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39893
I20260809 07:54:19.252525 1228 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:19.253536 1228 heartbeater.cc:507] Master 127.26.37.61:39893 requested a full tablet report, sending...
I20260809 07:54:19.253573 1227 heartbeater.cc:507] Master 127.26.37.60:33735 requested a full tablet report, sending...
I20260809 07:54:19.253608 1226 heartbeater.cc:507] Master 127.26.37.62:33383 requested a full tablet report, sending...
I20260809 07:54:19.256294 721 ts_manager.cc:194] Registered new tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:38797)
I20260809 07:54:19.256348 957 ts_manager.cc:194] Registered new tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:38797)
I20260809 07:54:19.256348 843 ts_manager.cc:194] Registered new tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:38797)
WARNING: no policy specified for kudu/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260809 07:54:19.258800 721 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:36103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.2@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.2@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.2 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:54:19.360785 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:54:19.457916 1031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "943334b0c66e48a7b966b6623474b72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:19.458282 1031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: LEADER
W20260809 07:54:19.726816 1244 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:19.727161 1244 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:19.727319 1244 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:19.736068 1244 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:19.736240 1244 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:54:19.746742 1244 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:19.748420 1244 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:19.749922 1244 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:19.760033 1252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:19.760070 1253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:19.760776 1255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:19.761449 1244 server_base.cc:1061] running on GCE node
I20260809 07:54:19.762064 1244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:19.763154 1244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:19.764361 1244 hybrid_clock.cc:648] HybridClock initialized: now 1786262059764226 us; error 106 us; skew 500 ppm
I20260809 07:54:19.764705 1244 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:19 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:19.768716 1244 init.cc:377] Logged in from keytab as kudu/127.26.37.2@KRBTEST.COM (short username kudu)
I20260809 07:54:19.774793 1244 webserver.cc:533] Webserver started at http://127.26.37.2:40623/ using document root <none> and password file <none>
I20260809 07:54:19.775558 1244 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:19.775707 1244 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:19.781105 1244 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:19.783897 1264 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:19.785493 1244 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:19.785666 1244 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "f710f7c65ed84ff5a00862072d6caf2b"
format_stamp: "Formatted at 2026-08-09 07:54:00 on dist-test-slave-bw88"
I20260809 07:54:19.786427 1244 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:19.805037 1244 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:19.808697 1244 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:19.809000 1244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:19.810039 1244 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:19.811657 1244 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:19.811789 1244 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:19.811913 1244 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:19.812007 1244 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:19.847272 1244 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:37957
I20260809 07:54:19.847357 1409 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:37957 every 8 connection(s)
I20260809 07:54:19.848924 1244 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:54:19.856370 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1244
I20260809 07:54:19.856642 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
Aug 09 07:54:19 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:54:19.881816 1410 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:19.882354 1410 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:19.883594 1410 heartbeater.cc:507] Master 127.26.37.62:33383 requested a full tablet report, sending...
Aug 09 07:54:19 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:54:19.885630 721 ts_manager.cc:194] Registered new tserver with Master: f710f7c65ed84ff5a00862072d6caf2b (127.26.37.2:37957)
I20260809 07:54:19.887054 721 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:58469
I20260809 07:54:19.896562 1412 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33735
I20260809 07:54:19.896749 1412 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:19.897217 1412 heartbeater.cc:507] Master 127.26.37.60:33735 requested a full tablet report, sending...
I20260809 07:54:19.898669 957 ts_manager.cc:194] Registered new tserver with Master: f710f7c65ed84ff5a00862072d6caf2b (127.26.37.2:37957)
Aug 09 07:54:19 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262059, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.2@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:54:19.906225 1413 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39893
I20260809 07:54:19.906409 1413 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:19.906679 1413 heartbeater.cc:507] Master 127.26.37.61:39893 requested a full tablet report, sending...
I20260809 07:54:19.907897 843 ts_manager.cc:194] Registered new tserver with Master: f710f7c65ed84ff5a00862072d6caf2b (127.26.37.2:37957)
WARNING: no policy specified for kudu/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.3@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.37.3@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.3 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:54:20.006911 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:54:20.261965 1226 heartbeater.cc:499] Master 127.26.37.62:33383 was elected leader, sending a full tablet report...
W20260809 07:54:20.373663 1433 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:20.373983 1433 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:20.374133 1433 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:20.382579 1433 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:20.382797 1433 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:54:20.394326 1433 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:20.396688 1433 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:20.398206 1433 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:20.407490 1442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:20.408236 1444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:20.409232 1449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:20.410135 1433 server_base.cc:1061] running on GCE node
I20260809 07:54:20.410709 1433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:20.411716 1433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:20.412932 1433 hybrid_clock.cc:648] HybridClock initialized: now 1786262060412794 us; error 109 us; skew 500 ppm
I20260809 07:54:20.413270 1433 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:20 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:20.416858 1433 init.cc:377] Logged in from keytab as kudu/127.26.37.3@KRBTEST.COM (short username kudu)
I20260809 07:54:20.418830 1433 webserver.cc:533] Webserver started at http://127.26.37.3:43605/ using document root <none> and password file <none>
I20260809 07:54:20.419587 1433 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:20.419734 1433 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:20.425379 1433 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:20.428376 1457 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:20.429956 1433 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:54:20.430140 1433 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "df79b59db14d45f1a6997dbb849af08e"
format_stamp: "Formatted at 2026-08-09 07:54:00 on dist-test-slave-bw88"
I20260809 07:54:20.431138 1433 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:20.451228 1433 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:20.461274 1433 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:20.461591 1433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:20.462563 1433 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:20.464159 1433 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:20.464244 1433 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:20.464329 1433 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:20.464383 1433 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:20.497893 1433 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:39805
I20260809 07:54:20.498008 1602 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:39805 every 8 connection(s)
I20260809 07:54:20.499578 1433 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:54:20.501225 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1433
I20260809 07:54:20.501488 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
Aug 09 07:54:20 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:54:20 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:54:20 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262060, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.3@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:54:20.540323 1605 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33735
I20260809 07:54:20.540756 1605 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:20.542372 1605 heartbeater.cc:507] Master 127.26.37.60:33735 requested a full tablet report, sending...
I20260809 07:54:20.542534 1603 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:20.542690 1603 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:20.543260 1603 heartbeater.cc:507] Master 127.26.37.62:33383 requested a full tablet report, sending...
I20260809 07:54:20.544363 957 ts_manager.cc:194] Registered new tserver with Master: df79b59db14d45f1a6997dbb849af08e (127.26.37.3:39805)
I20260809 07:54:20.545135 721 ts_manager.cc:194] Registered new tserver with Master: df79b59db14d45f1a6997dbb849af08e (127.26.37.3:39805)
I20260809 07:54:20.546169 721 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:33497
I20260809 07:54:20.549950 1606 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39893
I20260809 07:54:20.550091 1606 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:20.550348 1606 heartbeater.cc:507] Master 127.26.37.61:39893 requested a full tablet report, sending...
I20260809 07:54:20.551234 843 ts_manager.cc:194] Registered new tserver with Master: df79b59db14d45f1a6997dbb849af08e (127.26.37.3:39805)
I20260809 07:54:20.560909 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:54:20.568634 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:54:20.568830 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 677
I20260809 07:54:20.569829 766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:20.683156 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 677
W20260809 07:54:20.713189 939 connection.cc:573] server connection from 127.0.0.1:48092 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:20.714787 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33383
--webserver_interface=127.26.37.62
--webserver_port=38079
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:54:20.891096 1410 heartbeater.cc:499] Master 127.26.37.62:33383 was elected leader, sending a full tablet report...
W20260809 07:54:20.892831 1410 heartbeater.cc:646] Failed to heartbeat to 127.26.37.62:33383 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.37.62:33383: connect: Connection refused (error 111)
W20260809 07:54:21.074091 1620 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:21.074456 1620 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:21.074613 1620 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:21.082587 1620 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:21.082724 1620 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:21.082854 1620 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:21.082935 1620 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:21.095690 1620 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=38079
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:21.097313 1620 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:21.098795 1620 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:21.107089 1631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:21.107093 1632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:21.107573 1634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:21.108785 1620 server_base.cc:1061] running on GCE node
I20260809 07:54:21.109592 1620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:21.111120 1620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:21.112347 1620 hybrid_clock.cc:648] HybridClock initialized: now 1786262061112201 us; error 117 us; skew 500 ppm
I20260809 07:54:21.112685 1620 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:21 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:21.116230 1620 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:54:21.117795 1620 webserver.cc:533] Webserver started at http://127.26.37.62:38079/ using document root <none> and password file <none>
I20260809 07:54:21.118531 1620 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:21.118669 1620 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:21.124298 1620 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:54:21.127266 1642 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:21.128504 1620 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:54:21.128657 1620 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "943334b0c66e48a7b966b6623474b72f"
format_stamp: "Formatted at 2026-08-09 07:53:57 on dist-test-slave-bw88"
I20260809 07:54:21.129325 1620 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:21.141553 1620 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:21.145716 1620 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:21.145960 1620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:21.166379 1620 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33383
I20260809 07:54:21.166460 1709 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33383 every 8 connection(s)
I20260809 07:54:21.168058 1620 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:21.175215 1710 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:21.178141 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1620
I20260809 07:54:21.178505 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1044
I20260809 07:54:21.179044 1710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap starting.
Aug 09 07:54:21 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:54:21.197631 1219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:21.197788 1710 log.cc:826] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:21.209756 1710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:21.210457 1710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: Bootstrap complete.
I20260809 07:54:21.220553 1710 raft_consensus.cc:359] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:21.221541 1710 raft_consensus.cc:740] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943334b0c66e48a7b966b6623474b72f, State: Initialized, Role: FOLLOWER
I20260809 07:54:21.222363 1710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:21.223177 1717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:21.223439 1717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:21.224397 1710 sys_catalog.cc:565] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:21.234184 1729 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:54:21.234329 1729 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: loading cluster ID for follower catalog manager: success
I20260809 07:54:21.238309 1729 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: acquiring CA information for follower catalog manager: success
I20260809 07:54:21.240738 1729 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:21.278618 1620 master_runner.cc:264] No leader master found from master 943334b0c66e48a7b966b6623474b72f
I20260809 07:54:21.278776 1620 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:54:21.305794 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1044
W20260809 07:54:21.336553 942 connection.cc:573] server connection from 127.26.37.1:53763 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:21.336992 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:38797
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--webserver_port=44831
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:21.337450 1735 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:21.312937 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:43073 (local address 127.26.37.62:33383)
0809 07:54:21.313139 (+ 202us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:21.313143 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:21.336062 (+ 22919us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:43073: BlockingRecv error: recv got EOF from 127.26.37.1:43073 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 07:54:21.548766 1603 heartbeater.cc:499] Master 127.26.37.62:33383 was elected leader, sending a full tablet report...
I20260809 07:54:21.560299 1664 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "df79b59db14d45f1a6997dbb849af08e" instance_seqno: 1786262060489195) as {username='kudu', principal='kudu/127.26.37.3@KRBTEST.COM'} at 127.26.37.3:47871; Asking this server to re-register.
I20260809 07:54:21.561396 1603 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:21.563308 1664 ts_manager.cc:194] Registered new tserver with Master: df79b59db14d45f1a6997dbb849af08e (127.26.37.3:39805)
W20260809 07:54:21.705407 1741 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:21.705773 1741 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:21.705932 1741 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:21.714401 1741 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:21.714584 1741 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:21.725171 1741 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:38797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=44831
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:21.726882 1741 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:21.728451 1741 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:21.738505 1752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:21.738536 1750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:21.739290 1754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:21.739758 1741 server_base.cc:1061] running on GCE node
I20260809 07:54:21.740271 1741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:21.741468 1741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:21.742667 1741 hybrid_clock.cc:648] HybridClock initialized: now 1786262061742512 us; error 126 us; skew 500 ppm
I20260809 07:54:21.742995 1741 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:21 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:21.746680 1741 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:54:21.748377 1741 webserver.cc:533] Webserver started at http://127.26.37.1:44831/ using document root <none> and password file <none>
I20260809 07:54:21.749150 1741 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:21.749276 1741 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:21.754700 1741 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 07:54:21.757510 1763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:21.758818 1741 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:21.758987 1741 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:54:21.759768 1741 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:21.785230 1741 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:21.788978 1741 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:21.789286 1741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:21.790328 1741 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:21.792115 1741 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:21.792230 1741 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:21.792353 1741 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:21.792461 1741 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:21.828487 1741 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:38797
I20260809 07:54:21.828589 1906 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:38797 every 8 connection(s)
I20260809 07:54:21.830382 1741 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:21.831864 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1741
Aug 09 07:54:21 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:54:21 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:54:21 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:54:21.862089 1907 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:21.862578 1907 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:21.863775 1907 heartbeater.cc:507] Master 127.26.37.62:33383 requested a full tablet report, sending...
I20260809 07:54:21.866007 1664 ts_manager.cc:194] Registered new tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:38797)
Aug 09 07:54:21 dist-test-slave-bw88 krb5kdc[425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262061, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:54:21.868403 1910 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39893
I20260809 07:54:21.868546 1910 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:21.869084 1910 heartbeater.cc:507] Master 127.26.37.61:39893 requested a full tablet report, sending...
I20260809 07:54:21.870661 843 ts_manager.cc:194] Re-registered known tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:38797)
I20260809 07:54:21.875700 1908 heartbeater.cc:344] Connected to a master server at 127.26.37.60:33735
I20260809 07:54:21.875842 1908 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:21.876332 1908 heartbeater.cc:507] Master 127.26.37.60:33735 requested a full tablet report, sending...
I20260809 07:54:21.877575 957 ts_manager.cc:194] Re-registered known tserver with Master: f90dd2914ab54aaf9825ed855b6e1695 (127.26.37.1:38797)
I20260809 07:54:21.900311 1410 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33383
I20260809 07:54:21.900571 1410 heartbeater.cc:499] Master 127.26.37.62:33383 was elected leader, sending a full tablet report...
I20260809 07:54:21.901784 1664 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "f710f7c65ed84ff5a00862072d6caf2b" instance_seqno: 1786262059838676) as {username='kudu', principal='kudu/127.26.37.2@KRBTEST.COM'} at 127.26.37.2:57351; Asking this server to re-register.
I20260809 07:54:21.902565 1410 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:21.903739 1664 ts_manager.cc:194] Registered new tserver with Master: f710f7c65ed84ff5a00862072d6caf2b (127.26.37.2:37957)
I20260809 07:54:21.934937 1923 raft_consensus.cc:493] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 943334b0c66e48a7b966b6623474b72f)
I20260809 07:54:21.935228 1923 raft_consensus.cc:515] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:21.952414 1923 leader_election.cc:290] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 943334b0c66e48a7b966b6623474b72f (127.26.37.62:33383), 690c94c190be426bb6b79ee2f18b9166 (127.26.37.60:33735)
I20260809 07:54:21.954591 1677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de8b8e62a5234474993b7c1c26ab23f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "943334b0c66e48a7b966b6623474b72f" is_pre_election: true
I20260809 07:54:21.955224 1677 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de8b8e62a5234474993b7c1c26ab23f2 in term 2.
I20260809 07:54:21.955914 822 leader_election.cc:304] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 943334b0c66e48a7b966b6623474b72f, de8b8e62a5234474993b7c1c26ab23f2; no voters:
I20260809 07:54:21.956492 1923 raft_consensus.cc:2804] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:54:21.956691 1923 raft_consensus.cc:493] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 943334b0c66e48a7b966b6623474b72f)
I20260809 07:54:21.956879 1923 raft_consensus.cc:3060] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:54:21.960675 1923 raft_consensus.cc:515] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:21.961825 1923 leader_election.cc:290] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [CANDIDATE]: Term 3 election: Requested vote from peers 943334b0c66e48a7b966b6623474b72f (127.26.37.62:33383), 690c94c190be426bb6b79ee2f18b9166 (127.26.37.60:33735)
I20260809 07:54:21.962069 1677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de8b8e62a5234474993b7c1c26ab23f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "943334b0c66e48a7b966b6623474b72f"
I20260809 07:54:21.962357 1677 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:54:21.964916 970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de8b8e62a5234474993b7c1c26ab23f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "690c94c190be426bb6b79ee2f18b9166" is_pre_election: true
I20260809 07:54:21.965127 969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de8b8e62a5234474993b7c1c26ab23f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "690c94c190be426bb6b79ee2f18b9166"
I20260809 07:54:21.967835 1677 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de8b8e62a5234474993b7c1c26ab23f2 in term 3.
I20260809 07:54:21.968313 822 leader_election.cc:304] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 943334b0c66e48a7b966b6623474b72f, de8b8e62a5234474993b7c1c26ab23f2; no voters: 690c94c190be426bb6b79ee2f18b9166
I20260809 07:54:21.968708 1923 raft_consensus.cc:2804] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:54:21.969213 1923 raft_consensus.cc:697] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [term 3 LEADER]: Becoming Leader. State: Replica: de8b8e62a5234474993b7c1c26ab23f2, State: Running, Role: LEADER
I20260809 07:54:21.969715 1923 consensus_queue.cc:237] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:21.971519 1928 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de8b8e62a5234474993b7c1c26ab23f2. Latest consensus state: current_term: 3 leader_uuid: "de8b8e62a5234474993b7c1c26ab23f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:21.971807 1928 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:21.972532 1933 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:54:21.974591 1933 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:54:21.975634 1933 catalog_manager.cc:1295] Loaded cluster ID: 36520d2239f14b2682ce806b4ddb6d2f
I20260809 07:54:21.975766 1933 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:54:21.977218 1933 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:54:21.978379 1933 catalog_manager.cc:6103] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2: Loaded TSK: 0
I20260809 07:54:21.979563 1933 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:54:22.006397 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1741
W20260809 07:54:22.007849 1936 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:22.004010 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.26.37.1:38797)
0809 07:54:22.004808 (+ 798us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:22.004818 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:22.004858 (+ 40us) server_negotiation.cc:408] Connection header received
0809 07:54:22.004937 (+ 79us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:22.004944 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:22.005173 (+ 229us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:22.005408 (+ 235us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:22.006281 (+ 873us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58018: BlockingRecv error: recv got EOF from 127.0.0.1:58018 (error 108)
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":361,"threads_started":1}
W20260809 07:54:22.008641 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1741: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:38797: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:54:22.008773 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1741
I20260809 07:54:22.036250 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:38797
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--webserver_port=44831
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:54:22.184298 1940 raft_consensus.cc:493] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 943334b0c66e48a7b966b6623474b72f)
I20260809 07:54:22.184581 1940 raft_consensus.cc:515] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } }
I20260809 07:54:22.186450 1940 leader_election.cc:290] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 943334b0c66e48a7b966b6623474b72f (127.26.37.62:33383), de8b8e62a5234474993b7c1c26ab23f2 (127.26.37.61:39893)
I20260809 07:54:22.187294 856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "690c94c190be426bb6b79ee2f18b9166" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "de8b8e62a5234474993b7c1c26ab23f2" is_pre_election: true
I20260809 07:54:22.193951 1677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "690c94c190be426bb6b79ee2f18b9166" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "943334b0c66e48a7b966b6623474b72f" is_pre_election: true
I20260809 07:54:22.194299 1677 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 690c94c190be426bb6b79ee2f18b9166 in current term 3: Already voted for candidate de8b8e62a5234474993b7c1c26ab23f2 in this term.
I20260809 07:54:22.195052 941 leader_election.cc:304] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 690c94c190be426bb6b79ee2f18b9166; no voters: 943334b0c66e48a7b966b6623474b72f, de8b8e62a5234474993b7c1c26ab23f2
I20260809 07:54:22.195439 1940 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:54:22.197638 1940 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 07:54:22.410619 1939 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:22.410993 1939 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:22.411152 1939 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:54:22.411254 1939 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:54:22.415843 1677 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [term 3 FOLLOWER]: Refusing update from remote peer de8b8e62a5234474993b7c1c26ab23f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:54:22.416791 1928 consensus_queue.cc:1048] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:54:22.420790 1946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: New leader de8b8e62a5234474993b7c1c26ab23f2. Latest consensus state: current_term: 3 leader_uuid: "de8b8e62a5234474993b7c1c26ab23f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:22.421108 1946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:54:22.422112 1939 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
I20260809 07:54:22.422181 1928 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "de8b8e62a5234474993b7c1c26ab23f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:22.422454 1928 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: LEADER
W20260809 07:54:22.422523 1939 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:22.423507 1946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "de8b8e62a5234474993b7c1c26ab23f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:22.423695 1946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943334b0c66e48a7b966b6623474b72f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:22.424180 969 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [term 3 FOLLOWER]: Refusing update from remote peer de8b8e62a5234474993b7c1c26ab23f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 07:54:22.424890 1928 consensus_queue.cc:1048] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:54:22.427685 1958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "de8b8e62a5234474993b7c1c26ab23f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:22.428084 1958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:22.428011 1940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de8b8e62a5234474993b7c1c26ab23f2. Latest consensus state: current_term: 3 leader_uuid: "de8b8e62a5234474993b7c1c26ab23f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:22.428270 1940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690c94c190be426bb6b79ee2f18b9166 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:22.437211 1939 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:38797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=44831
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:22.439041 1939 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:22.440546 1939 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:22.450289 1962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:22.450258 1960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:22.451202 1966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:22.451632 1939 server_base.cc:1061] running on GCE node
I20260809 07:54:22.452230 1939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:22.453290 1939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:22.454564 1939 hybrid_clock.cc:648] HybridClock initialized: now 1786262062454426 us; error 83 us; skew 500 ppm
I20260809 07:54:22.454867 1939 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:22 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262062, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:22.458485 1939 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:54:22.460240 1939 webserver.cc:533] Webserver started at http://127.26.37.1:44831/ using document root <none> and password file <none>
I20260809 07:54:22.461097 1939 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:22.461238 1939 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:22.466681 1939 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 07:54:22.469427 1974 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:22.470853 1939 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:22.471032 1939 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:54:22.471711 1939 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:22.484799 1939 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:22.488349 1939 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:22.488660 1939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:22.489679 1939 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:22.491442 1939 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:22.491515 1939 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:22.491698 1939 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:22.491768 1939 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:54:22.492167 1939 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:54:22.492251 1939 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:54:22.492340 1939 fault_injection.cc:59] injecting fault for kudu (pid 1939): FLAGS_fault_before_start (process will exit)
I20260809 07:54:22.521579 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:38797
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--webserver_port=44831
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:54:22.889762 1988 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:22.890118 1988 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:22.890272 1988 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:22.898814 1988 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:22.899056 1988 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:22.911789 1988 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35311,127.26.37.21:37485,127.26.37.22:43163,127.26.37.23:34499,127.26.37.24:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:38797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=44831
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33383,127.26.37.61:39893,127.26.37.60:33735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:22.913734 1988 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:22.915192 1988 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:22.924801 1995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:22.924861 1996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:22.926497 1999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:22.927278 1988 server_base.cc:1061] running on GCE node
I20260809 07:54:22.927975 1988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:22.929139 1988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:22.930363 1988 hybrid_clock.cc:648] HybridClock initialized: now 1786262062930200 us; error 131 us; skew 500 ppm
I20260809 07:54:22.930708 1988 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:54:22 dist-test-slave-bw88 krb5kdc[425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262062, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:54:22.934376 1988 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:54:22.936069 1988 webserver.cc:533] Webserver started at http://127.26.37.1:44831/ using document root <none> and password file <none>
I20260809 07:54:22.936890 1988 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:22.937011 1988 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:22.942570 1988 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:54:22.945259 2010 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:22.946673 1988 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:22.946825 1988 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f90dd2914ab54aaf9825ed855b6e1695"
format_stamp: "Formatted at 2026-08-09 07:53:59 on dist-test-slave-bw88"
I20260809 07:54:22.947502 1988 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:22.964565 1988 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:22.968464 1988 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:22.968711 1988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:22.969743 1988 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:22.971328 1988 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:22.971421 1988 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:22.971506 1988 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:22.971558 1988 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:22.977859 1928 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "de8b8e62a5234474993b7c1c26ab23f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943334b0c66e48a7b966b6623474b72f" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33383 } } peers { permanent_uuid: "de8b8e62a5234474993b7c1c26ab23f2" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39893 } } peers { permanent_uuid: "690c94c190be426bb6b79ee2f18b9166" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 33735 } } }
I20260809 07:54:22.978181 1928 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8b8e62a5234474993b7c1c26ab23f2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:23.009157 1988 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:38797
I20260809 07:54:23.009244 2163 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:38797 every 8 connection(s)
I20260809 07:54:23.010946 1988 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:23.017554 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 1988
I20260809 07:54:23.017962 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1988
W20260809 07:54:23.041242 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1988: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:38797: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:54:23.041482 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1988
I20260809 07:54:23.072933 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1244
W20260809 07:54:23.073140 2173 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:23.015149 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:42583 (local address 127.26.37.62:33383)
0809 07:54:23.031382 (+ 16233us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:23.031387 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:23.031405 (+ 18us) server_negotiation.cc:408] Connection header received
0809 07:54:23.031471 (+ 66us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:23.031475 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:23.031563 (+ 88us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:23.031668 (+ 105us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:23.034769 (+ 3101us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:54:23.036747 (+ 1978us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:54:23.039244 (+ 2497us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:54:23.039908 (+ 664us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:54:23.072786 (+ 32878us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:42583: BlockingRecv error: recv got EOF from 127.26.37.1:42583 (error 108)
Metrics: {"server-negotiator.queue_time_us":16116,"thread_start_us":15988,"threads_started":1}
W20260809 07:54:23.073776 2174 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:23.015382 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:50299 (local address 127.26.37.60:33735)
0809 07:54:23.047852 (+ 32470us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:23.047860 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:23.047884 (+ 24us) server_negotiation.cc:408] Connection header received
0809 07:54:23.072658 (+ 24774us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:50299: BlockingRecv error: recv got EOF from 127.26.37.1:50299 (error 108)
Metrics: {"server-negotiator.queue_time_us":32329,"thread_start_us":38026,"threads_started":1}
W20260809 07:54:23.074790 2179 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:23.040898 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:51419 (local address 127.26.37.61:39893)
0809 07:54:23.053938 (+ 13040us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:23.053943 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:23.073221 (+ 19278us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:51419: BlockingRecv error: recv got EOF from 127.26.37.1:51419 (error 108)
Metrics: {"server-negotiator.queue_time_us":12913,"thread_start_us":12804,"threads_started":1}
W20260809 07:54:23.076089 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1244: Not authorized: Client connection negotiation failed: client connection to 127.26.37.2:37957: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:54:23.076256 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1244
I20260809 07:54:23.103119 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1433
W20260809 07:54:23.106302 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1433: Not authorized: Client connection negotiation failed: client connection to 127.26.37.3:39805: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:54:23.106431 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1433
I20260809 07:54:23.131130 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1620
W20260809 07:54:23.133284 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 1620: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:33383: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:54:23.133474 2173 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:23.132011 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.26.37.62:33383)
0809 07:54:23.132201 (+ 190us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:23.132206 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:23.132224 (+ 18us) server_negotiation.cc:408] Connection header received
0809 07:54:23.132415 (+ 191us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:23.132420 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:23.132487 (+ 67us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:23.132581 (+ 94us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:23.133214 (+ 633us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49480: BlockingRecv error: recv got EOF from 127.0.0.1:49480 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 07:54:23.133508 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 1620
I20260809 07:54:23.160934 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 800
I20260809 07:54:23.162309 879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:23.276490 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 800
I20260809 07:54:23.305451 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 915
I20260809 07:54:23.306700 994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:23.417223 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 915
Shutting down hive metastore.
2026-08-09T07:54:23Z chronyd exiting
2026-08-09T07:54:23Z chronyd exiting
2026-08-09T07:54:23Z chronyd exiting
2026-08-09T07:54:23Z chronyd exiting
2026-08-09T07:54:23Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/11 (46182 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/0
I20260809 07:54:23.896138 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:32907
--webserver_interface=127.26.37.62
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543 with env {}
W20260809 07:54:24.259081 2195 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:24.259498 2195 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:24.259656 2195 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:24.268373 2195 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:24.268513 2195 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:24.268646 2195 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:24.268785 2195 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:24.279678 2195 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:24.281567 2195 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:24.283530 2195 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:24.292158 2200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:24.292131 2202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:24.292994 2204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:24.293658 2195 server_base.cc:1061] running on GCE node
I20260809 07:54:24.294450 2195 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:24.294665 2195 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:24.294799 2195 hybrid_clock.cc:648] HybridClock initialized: now 1786262064294774 us; error 0 us; skew 500 ppm
I20260809 07:54:24.295214 2195 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:24.297997 2195 webserver.cc:533] Webserver started at http://127.26.37.62:42621/ using document root <none> and password file <none>
I20260809 07:54:24.298836 2195 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:24.298990 2195 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:24.299393 2195 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:24.302011 2195 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "3fe198c78a81422b9d5890e314b6cc6a"
format_stamp: "Formatted at 2026-08-09 07:54:24 on dist-test-slave-bw88"
I20260809 07:54:24.302889 2195 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "3fe198c78a81422b9d5890e314b6cc6a"
format_stamp: "Formatted at 2026-08-09 07:54:24 on dist-test-slave-bw88"
I20260809 07:54:24.308632 2195 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:54:24.312230 2212 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:24.313627 2195 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:54:24.313822 2195 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "3fe198c78a81422b9d5890e314b6cc6a"
format_stamp: "Formatted at 2026-08-09 07:54:24 on dist-test-slave-bw88"
I20260809 07:54:24.314054 2195 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:24.346067 2195 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:24.347123 2195 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:24.347452 2195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:24.371757 2195 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:32907
I20260809 07:54:24.371778 2284 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:32907 every 8 connection(s)
I20260809 07:54:24.373682 2195 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:24.377996 2285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:24.380959 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2195
I20260809 07:54:24.381215 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:54:24.382890 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:43021
--webserver_interface=127.26.37.61
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543 with env {}
I20260809 07:54:24.385844 2285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:24.403855 2285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:24.405160 2215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43021: connect: Connection refused (error 111)
W20260809 07:54:24.406790 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43021: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43021: connect: Connection refused (error 111)
I20260809 07:54:24.456225 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } attempt: 1
W20260809 07:54:24.458410 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43021: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43021: connect: Connection refused (error 111)
I20260809 07:54:24.526695 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } attempt: 2
W20260809 07:54:24.528755 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43021: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43021: connect: Connection refused (error 111)
I20260809 07:54:24.620055 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } attempt: 3
W20260809 07:54:24.622161 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43021: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43021: connect: Connection refused (error 111)
W20260809 07:54:24.741173 2292 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:24.741575 2292 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:24.741712 2292 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:24.749679 2292 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:24.749795 2292 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:24.749939 2292 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:24.750020 2292 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:24.759531 2292 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:43021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:24.761258 2292 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:24.763091 2292 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:54:24.765434 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } attempt: 4
W20260809 07:54:24.767568 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43021: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43021: connect: Connection refused (error 111)
W20260809 07:54:24.771615 2299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:24.771919 2300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:24.772318 2303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:24.773478 2292 server_base.cc:1061] running on GCE node
I20260809 07:54:24.774207 2292 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:24.774392 2292 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:24.774492 2292 hybrid_clock.cc:648] HybridClock initialized: now 1786262064774469 us; error 0 us; skew 500 ppm
I20260809 07:54:24.774835 2292 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:24.781375 2292 webserver.cc:533] Webserver started at http://127.26.37.61:39301/ using document root <none> and password file <none>
I20260809 07:54:24.782167 2292 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:24.782291 2292 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:24.782656 2292 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:24.785048 2292 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "2bb76722ceea4534b94f93192d4c2295"
format_stamp: "Formatted at 2026-08-09 07:54:24 on dist-test-slave-bw88"
I20260809 07:54:24.785946 2292 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "2bb76722ceea4534b94f93192d4c2295"
format_stamp: "Formatted at 2026-08-09 07:54:24 on dist-test-slave-bw88"
I20260809 07:54:24.791360 2292 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:54:24.794804 2311 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:24.796222 2292 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 07:54:24.796401 2292 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "2bb76722ceea4534b94f93192d4c2295"
format_stamp: "Formatted at 2026-08-09 07:54:24 on dist-test-slave-bw88"
I20260809 07:54:24.796633 2292 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:24.835626 2292 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:24.836611 2292 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:24.836906 2292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:24.858044 2292 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:43021
I20260809 07:54:24.858070 2377 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:43021 every 8 connection(s)
I20260809 07:54:24.859660 2292 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:54:24.860090 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2292
I20260809 07:54:24.860306 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:54:24.861574 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:36543
--webserver_interface=127.26.37.60
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543 with env {}
I20260809 07:54:24.864341 2378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:24.870291 2378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:24.884193 2378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:24.894155 2378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:24.895370 2317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
W20260809 07:54:24.897205 2378 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
I20260809 07:54:24.946684 2378 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 1
W20260809 07:54:24.949033 2378 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
I20260809 07:54:25.017309 2378 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 2
W20260809 07:54:25.019529 2378 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
I20260809 07:54:25.066839 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } attempt: 5
I20260809 07:54:25.073623 2285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:25.075709 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
I20260809 07:54:25.110814 2378 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 3
W20260809 07:54:25.113243 2378 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
I20260809 07:54:25.126924 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 1
W20260809 07:54:25.128916 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
I20260809 07:54:25.197167 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 2
W20260809 07:54:25.199611 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
I20260809 07:54:25.256575 2378 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 4
W20260809 07:54:25.258579 2378 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
W20260809 07:54:25.262351 2381 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:25.262667 2381 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:25.262809 2381 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:25.271100 2381 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:25.271178 2381 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:25.271287 2381 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:25.271375 2381 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:25.281037 2381 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:36543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:25.282699 2381 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:25.284585 2381 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:25.293083 2393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:25.293006 2394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:25.294142 2400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:25.294761 2381 server_base.cc:1061] running on GCE node
I20260809 07:54:25.295459 2381 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:25.295605 2381 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:25.295670 2381 hybrid_clock.cc:648] HybridClock initialized: now 1786262065295653 us; error 0 us; skew 500 ppm
I20260809 07:54:25.295944 2381 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:25.298424 2381 webserver.cc:533] Webserver started at http://127.26.37.60:43677/ using document root <none> and password file <none>
I20260809 07:54:25.299111 2381 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:25.299206 2381 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:25.299508 2381 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:25.301890 2381 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "6448dc3cba744fe5a58f50957bedc339"
format_stamp: "Formatted at 2026-08-09 07:54:25 on dist-test-slave-bw88"
I20260809 07:54:25.302670 2381 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "6448dc3cba744fe5a58f50957bedc339"
format_stamp: "Formatted at 2026-08-09 07:54:25 on dist-test-slave-bw88"
I20260809 07:54:25.305859 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 3
W20260809 07:54:25.307888 2285 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36543: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36543: connect: Connection refused (error 111)
I20260809 07:54:25.308176 2381 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 07:54:25.311460 2406 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:25.312801 2381 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:54:25.312983 2381 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "6448dc3cba744fe5a58f50957bedc339"
format_stamp: "Formatted at 2026-08-09 07:54:25 on dist-test-slave-bw88"
I20260809 07:54:25.313182 2381 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:25.336692 2381 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:25.337759 2381 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:25.338088 2381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:25.360954 2381 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:36543
I20260809 07:54:25.361001 2473 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:36543 every 8 connection(s)
I20260809 07:54:25.362607 2381 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:54:25.367973 2474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:25.371726 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2381
I20260809 07:54:25.371930 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:54:25.375685 2474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:25.398233 2474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:25.404968 2474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:25.416290 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: Bootstrap starting.
I20260809 07:54:25.419786 2474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:25.421244 2474 log.cc:826] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:25.423786 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: No bootstrap required, opened a new log
I20260809 07:54:25.429474 2474 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.429772 2474 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:25.429873 2474 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6448dc3cba744fe5a58f50957bedc339, State: Initialized, Role: FOLLOWER
I20260809 07:54:25.430446 2474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.431324 2486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.431631 2486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:25.432456 2474 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:54:25.442675 2502 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:25.442898 2502 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:25.439972 2381 master_runner.cc:187] Error getting master registration for 127.26.37.62:32907: OK, instance_id { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" instance_seqno: 1786262064365633 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:54:25.447952 2381 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:54:25.485140 2285 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 4
I20260809 07:54:25.507601 2285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: Bootstrap starting.
I20260809 07:54:25.512473 2285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:25.514127 2285 log.cc:826] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:25.517503 2285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: No bootstrap required, opened a new log
I20260809 07:54:25.525830 2285 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.526146 2285 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:25.526290 2285 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe198c78a81422b9d5890e314b6cc6a, State: Initialized, Role: FOLLOWER
I20260809 07:54:25.526948 2285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.527983 2505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.528299 2505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:25.529181 2285 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:54:25.539726 2526 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:25.539999 2526 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:25.539743 2195 master_runner.cc:187] Error getting master registration for 127.26.37.61:43021: OK, instance_id { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" instance_seqno: 1786262064852403 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:54:25.546994 2195 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:54:25.557829 2378 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } attempt: 5
I20260809 07:54:25.573102 2378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295: Bootstrap starting.
I20260809 07:54:25.577792 2378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:25.579332 2378 log.cc:826] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:25.583101 2378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295: No bootstrap required, opened a new log
W20260809 07:54:25.590377 2412 tablet.cc:2406] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:54:25.591459 2378 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.591823 2378 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:25.592020 2378 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bb76722ceea4534b94f93192d4c2295, State: Initialized, Role: FOLLOWER
I20260809 07:54:25.592794 2378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.593912 2529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.594296 2529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:25.595306 2378 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:54:25.600425 2220 tablet.cc:2406] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:54:25.615760 2292 master_runner.cc:264] No leader master found from master 2bb76722ceea4534b94f93192d4c2295
I20260809 07:54:25.615933 2292 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:54:25.620143 2543 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:25.620419 2543 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:25.643715 2529 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:54:25.644026 2529 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.646498 2529 leader_election.cc:290] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fe198c78a81422b9d5890e314b6cc6a (127.26.37.62:32907), 6448dc3cba744fe5a58f50957bedc339 (127.26.37.60:36543)
I20260809 07:54:25.647853 2251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2bb76722ceea4534b94f93192d4c2295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe198c78a81422b9d5890e314b6cc6a" is_pre_election: true
I20260809 07:54:25.648481 2251 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bb76722ceea4534b94f93192d4c2295 in term 0.
I20260809 07:54:25.649959 2440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2bb76722ceea4534b94f93192d4c2295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6448dc3cba744fe5a58f50957bedc339" is_pre_election: true
I20260809 07:54:25.650501 2440 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bb76722ceea4534b94f93192d4c2295 in term 0.
I20260809 07:54:25.651674 2312 leader_election.cc:304] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bb76722ceea4534b94f93192d4c2295, 3fe198c78a81422b9d5890e314b6cc6a; no voters:
I20260809 07:54:25.657140 2529 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:54:25.657325 2529 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:54:25.657648 2529 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:25.661330 2529 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.662755 2251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2bb76722ceea4534b94f93192d4c2295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe198c78a81422b9d5890e314b6cc6a"
I20260809 07:54:25.662997 2251 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:25.666793 2251 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bb76722ceea4534b94f93192d4c2295 in term 1.
I20260809 07:54:25.667436 2312 leader_election.cc:304] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bb76722ceea4534b94f93192d4c2295, 3fe198c78a81422b9d5890e314b6cc6a; no voters:
I20260809 07:54:25.671842 2440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2bb76722ceea4534b94f93192d4c2295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6448dc3cba744fe5a58f50957bedc339"
I20260809 07:54:25.672140 2440 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:25.676015 2440 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bb76722ceea4534b94f93192d4c2295 in term 1.
I20260809 07:54:25.681644 2529 leader_election.cc:290] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [CANDIDATE]: Term 1 election: Requested vote from peers 3fe198c78a81422b9d5890e314b6cc6a (127.26.37.62:32907), 6448dc3cba744fe5a58f50957bedc339 (127.26.37.60:36543)
I20260809 07:54:25.682145 2529 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:54:25.682654 2529 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 1 LEADER]: Becoming Leader. State: Replica: 2bb76722ceea4534b94f93192d4c2295, State: Running, Role: LEADER
I20260809 07:54:25.683523 2529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } }
I20260809 07:54:25.695502 2545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb76722ceea4534b94f93192d4c2295. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.696000 2545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:25.708144 2547 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:54:25.710850 2547 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:54:25.727025 2440 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 1 FOLLOWER]: Refusing update from remote peer 2bb76722ceea4534b94f93192d4c2295: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:54:25.728348 2545 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:54:25.740629 2251 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 1 FOLLOWER]: Refusing update from remote peer 2bb76722ceea4534b94f93192d4c2295: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:54:25.741830 2545 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:54:25.781723 2505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb76722ceea4534b94f93192d4c2295. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.782061 2505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:25.787003 2548 mvcc.cc:204] Tried to move back new op lower bound from 7316529421204221952 to 7316529421052657664. Current Snapshot: MvccSnapshot[applied={T|T < 7316529421204221952}]
I20260809 07:54:25.790356 2550 mvcc.cc:204] Tried to move back new op lower bound from 7316529421204221952 to 7316529421052657664. Current Snapshot: MvccSnapshot[applied={T|T < 7316529421204221952}]
I20260809 07:54:25.798666 2486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb76722ceea4534b94f93192d4c2295. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.799170 2486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:25.804816 2529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.805126 2529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:25.807194 2505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.807461 2505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:25.810801 2545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.811509 2545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:25.812301 2549 mvcc.cc:204] Tried to move back new op lower bound from 7316529421204221952 to 7316529421052657664. Current Snapshot: MvccSnapshot[applied={T|T < 7316529421204221952}]
I20260809 07:54:25.815766 2547 catalog_manager.cc:1383] Generated new cluster ID: 3737cb6c87e04f83a6af2f245644c995
I20260809 07:54:25.815898 2547 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:54:25.832376 2486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } }
I20260809 07:54:25.832613 2486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:25.847677 2547 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:54:25.849560 2547 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:54:25.913278 2547 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295: Generated new TSK 0
I20260809 07:54:25.914597 2547 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:54:25.935571 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--time_source=system_unsync with env {}
W20260809 07:54:26.383332 2560 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:26.383708 2560 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:26.383818 2560 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:26.396889 2560 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:26.397113 2560 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:26.408586 2560 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:26.410457 2560 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:26.412375 2560 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:26.422149 2566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:26.422191 2568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:26.423069 2573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:26.423524 2560 server_base.cc:1061] running on GCE node
I20260809 07:54:26.424057 2560 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:26.424247 2560 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:26.424345 2560 hybrid_clock.cc:648] HybridClock initialized: now 1786262066424324 us; error 0 us; skew 500 ppm
I20260809 07:54:26.424660 2560 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:26.427044 2560 webserver.cc:533] Webserver started at http://127.26.37.1:39489/ using document root <none> and password file <none>
I20260809 07:54:26.427786 2560 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:26.427922 2560 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:26.428314 2560 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:26.431087 2560 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "8064ba99193a4fb4957ac68f458a4983"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:26.431891 2560 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "8064ba99193a4fb4957ac68f458a4983"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:26.437529 2560 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 07:54:26.442389 2582 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:26.443663 2560 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:54:26.443871 2560 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "8064ba99193a4fb4957ac68f458a4983"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:26.444095 2560 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:26.444841 2502 catalog_manager.cc:1295] Loaded cluster ID: 3737cb6c87e04f83a6af2f245644c995
I20260809 07:54:26.444921 2502 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: loading cluster ID for follower catalog manager: success
I20260809 07:54:26.447448 2502 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: acquiring CA information for follower catalog manager: success
I20260809 07:54:26.449115 2502 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:26.462102 2560 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:26.462934 2560 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:26.463223 2560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:26.464169 2560 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:26.465849 2560 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:26.465983 2560 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:26.466109 2560 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:26.466203 2560 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:26.502797 2560 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44127
I20260809 07:54:26.502856 2738 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44127 every 8 connection(s)
I20260809 07:54:26.504335 2560 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:26.509977 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2560
I20260809 07:54:26.510306 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:54:26.535880 2741 heartbeater.cc:344] Connected to a master server at 127.26.37.61:43021
I20260809 07:54:26.535992 2739 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36543
I20260809 07:54:26.536047 2740 heartbeater.cc:344] Connected to a master server at 127.26.37.62:32907
I20260809 07:54:26.536540 2741 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:26.536533 2740 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:26.536530 2739 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:26.537783 2740 heartbeater.cc:507] Master 127.26.37.62:32907 requested a full tablet report, sending...
I20260809 07:54:26.537806 2739 heartbeater.cc:507] Master 127.26.37.60:36543 requested a full tablet report, sending...
I20260809 07:54:26.538601 2741 heartbeater.cc:507] Master 127.26.37.61:43021 requested a full tablet report, sending...
I20260809 07:54:26.540191 2334 ts_manager.cc:194] Registered new tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:26.540490 2237 ts_manager.cc:194] Registered new tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:26.542145 2526 catalog_manager.cc:1295] Loaded cluster ID: 3737cb6c87e04f83a6af2f245644c995
I20260809 07:54:26.542126 2334 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:42989
I20260809 07:54:26.542218 2526 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: loading cluster ID for follower catalog manager: success
I20260809 07:54:26.543746 2424 ts_manager.cc:194] Registered new tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:26.545300 2526 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: acquiring CA information for follower catalog manager: success
I20260809 07:54:26.547544 2526 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:26.550844 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:26.553618 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:32839
--webserver_interface=127.26.37.59
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839 with env {}
W20260809 07:54:26.917615 2752 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:26.917989 2752 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:26.918133 2752 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:26.926626 2752 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:26.926720 2752 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:26.926821 2752 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:26.926877 2752 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:26.937124 2752 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:32839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:26.939050 2752 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:26.940964 2752 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:26.949394 2758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:26.949357 2757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:26.950642 2762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:26.950984 2752 server_base.cc:1061] running on GCE node
I20260809 07:54:26.951776 2752 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:26.951924 2752 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:26.951989 2752 hybrid_clock.cc:648] HybridClock initialized: now 1786262066951971 us; error 0 us; skew 500 ppm
I20260809 07:54:26.952261 2752 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:26.954945 2752 webserver.cc:533] Webserver started at http://127.26.37.59:46157/ using document root <none> and password file <none>
I20260809 07:54:26.955633 2752 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:26.955721 2752 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:26.956130 2752 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:26.958441 2752 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "5c5bd434943444f6be029c425b85c047"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:26.959156 2752 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "5c5bd434943444f6be029c425b85c047"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:26.964332 2752 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:54:26.967638 2769 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:26.968920 2752 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:54:26.969113 2752 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "5c5bd434943444f6be029c425b85c047"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:26.969319 2752 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:27.011220 2752 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:27.012292 2752 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:27.012591 2752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:27.033923 2752 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:32839
I20260809 07:54:27.033946 2835 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:32839 every 8 connection(s)
I20260809 07:54:27.035907 2752 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:54:27.037602 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2752
I20260809 07:54:27.041524 2837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:27.047628 2837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:27.058805 2837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:27.065217 2837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:27.071234 2837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:27.080847 2837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Bootstrap starting.
I20260809 07:54:27.084079 2837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:27.085412 2837 log.cc:826] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:27.089126 2837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: No bootstrap required, opened a new log
I20260809 07:54:27.095573 2837 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } }
I20260809 07:54:27.095912 2837 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:27.096017 2837 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5bd434943444f6be029c425b85c047, State: Initialized, Role: FOLLOWER
I20260809 07:54:27.096753 2837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } }
I20260809 07:54:27.097718 2851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } } }
I20260809 07:54:27.098022 2851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:27.098737 2837 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:27.107481 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2560
W20260809 07:54:27.109540 2864 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:27.109673 2864 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:27.117899 2752 master_runner.cc:429] Detected that this master 5c5bd434943444f6be029c425b85c047 is joining an existing cluster
I20260809 07:54:27.117940 2731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:27.118135 2752 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:54:27.119577 2752 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:54:27.120151 2752 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:54:27.120467 2752 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: stopping tablet replica
I20260809 07:54:27.122901 2752 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:32839
I20260809 07:54:27.173257 2334 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:32839: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5c5bd434943444f6be029c425b85c047"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 32839
}
attrs {
promote: true
}
}
dest_uuid: "2bb76722ceea4534b94f93192d4c2295"
cas_config_opid_index: -1
I20260809 07:54:27.175742 2334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } }
I20260809 07:54:27.179488 2250 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 1 FOLLOWER]: Refusing update from remote peer 2bb76722ceea4534b94f93192d4c2295: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260809 07:54:27.179819 2315 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 -> Peer 5c5bd434943444f6be029c425b85c047 (127.26.37.59:32839): Couldn't send request to peer 5c5bd434943444f6be029c425b85c047. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:54:27.180212 2545 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:54:27.181756 2439 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 1 FOLLOWER]: Refusing update from remote peer 2bb76722ceea4534b94f93192d4c2295: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:54:27.182381 2545 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:54:27.191610 2887 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5c5bd434943444f6be029c425b85c047 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.191841 2250 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5c5bd434943444f6be029c425b85c047 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.196137 2887 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:32839
I20260809 07:54:27.199082 2752 master.cc:562] Master@127.26.37.59:32839 shutting down...
I20260809 07:54:27.205648 2439 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5c5bd434943444f6be029c425b85c047 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.209991 2888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.210405 2888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:27.210307 2545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.210671 2545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:27.211381 2889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.211848 2889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:27.209870 2887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.220937 2887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:27.226076 2752 master.cc:584] Master@127.26.37.59:32839 shutdown complete.
I20260809 07:54:27.226269 2752 master_runner.cc:306] Clearing existing system tablet
I20260809 07:54:27.226843 2752 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:27.226986 2752 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:27.229305 2752 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:54:27.230618 2895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.230939 2895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:27.233691 2897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:27.234140 2752 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:27.234452 2752 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260809 07:54:27.234587 2752 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "5c5bd434943444f6be029c425b85c047"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:27.238186 2752 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:54:27.243067 2752 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:54:27.243237 2752 log.cc:1199] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:54:27.243695 2752 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Deleting consensus metadata
I20260809 07:54:27.243947 2752 master_runner.cc:316] Copying system tablet from 127.26.37.61:43021
I20260809 07:54:27.247038 2752 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.61:43021
I20260809 07:54:27.254972 2359 tablet_copy_service.cc:140] P 2bb76722ceea4534b94f93192d4c2295: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5c5bd434943444f6be029c425b85c047 ({username='slave'} at 127.0.0.1:50844)
I20260809 07:54:27.255379 2359 tablet_copy_service.cc:161] P 2bb76722ceea4534b94f93192d4c2295: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5c5bd434943444f6be029c425b85c047 at {username='slave'} at 127.0.0.1:50844: session id = 5c5bd434943444f6be029c425b85c047-00000000000000000000000000000000
I20260809 07:54:27.258692 2359 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:54:27.261224 2752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:54:27.266161 2752 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: tablet copy: Starting download of 0 data blocks...
I20260809 07:54:27.266579 2752 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: tablet copy: Starting download of 1 WAL segments...
I20260809 07:54:27.270097 2752 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:54:27.273226 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2560
I20260809 07:54:27.279438 2359 tablet_copy_service.cc:342] P 2bb76722ceea4534b94f93192d4c2295: Request end of tablet copy session 5c5bd434943444f6be029c425b85c047-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:50844
I20260809 07:54:27.279747 2359 tablet_copy_service.cc:434] P 2bb76722ceea4534b94f93192d4c2295: ending tablet copy session 5c5bd434943444f6be029c425b85c047-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5c5bd434943444f6be029c425b85c047
I20260809 07:54:27.291829 2752 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:27.294167 2905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:27.300546 2906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:27.304196 2910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:27.305632 2752 server_base.cc:1061] running on GCE node
I20260809 07:54:27.305968 2752 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:27.306095 2752 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:27.306165 2752 hybrid_clock.cc:648] HybridClock initialized: now 1786262067306142 us; error 0 us; skew 500 ppm
I20260809 07:54:27.306380 2752 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:27.307266 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44127
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--webserver_port=39489
--webserver_interface=127.26.37.1
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839 with env {}
I20260809 07:54:27.307839 2752 webserver.cc:533] Webserver started at http://127.26.37.59:39975/ using document root <none> and password file <none>
I20260809 07:54:27.308198 2752 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:27.308295 2752 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:27.310729 2752 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:54:27.312529 2915 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:27.312961 2752 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 07:54:27.313094 2752 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "5c5bd434943444f6be029c425b85c047"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:27.313261 2752 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:27.335584 2752 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:27.336319 2752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:27.415148 2752 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:32839
I20260809 07:54:27.415176 2991 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:32839 every 8 connection(s)
I20260809 07:54:27.416173 2752 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:54:27.423499 2992 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:27.426260 2992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Bootstrap starting.
I20260809 07:54:27.439889 2992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:27.440387 2992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: Bootstrap complete.
I20260809 07:54:27.441309 2992 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } }
I20260809 07:54:27.441594 2992 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5c5bd434943444f6be029c425b85c047, State: Initialized, Role: LEARNER
I20260809 07:54:27.441905 2992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } }
I20260809 07:54:27.445878 2998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.446189 2992 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:27.446238 2998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:54:27.457113 3013 catalog_manager.cc:1295] Loaded cluster ID: 3737cb6c87e04f83a6af2f245644c995
I20260809 07:54:27.457276 3013 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: loading cluster ID for follower catalog manager: success
I20260809 07:54:27.460951 3013 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: acquiring CA information for follower catalog manager: success
I20260809 07:54:27.463466 3013 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:27.572773 2956 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:54:27.573793 2998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb76722ceea4534b94f93192d4c2295. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:27.574126 2998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: This master's current role is: LEARNER
W20260809 07:54:27.729434 2921 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:27.729802 2921 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:27.729940 2921 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:27.738735 2921 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:27.738958 2921 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:27.749123 2921 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44127
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39489
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:27.750896 2921 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:27.752910 2921 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:27.763058 3019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:27.763137 3020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:27.763721 3023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:27.764088 2921 server_base.cc:1061] running on GCE node
I20260809 07:54:27.764648 2921 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:27.764832 2921 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:27.764930 2921 hybrid_clock.cc:648] HybridClock initialized: now 1786262067764893 us; error 0 us; skew 500 ppm
I20260809 07:54:27.765249 2921 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:27.767531 2921 webserver.cc:533] Webserver started at http://127.26.37.1:39489/ using document root <none> and password file <none>
I20260809 07:54:27.768328 2921 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:27.768473 2921 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:27.774080 2921 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:54:27.776973 3030 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:27.778465 2921 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:27.778676 2921 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "8064ba99193a4fb4957ac68f458a4983"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:27.779422 2921 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:27.800185 2921 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:27.801088 2921 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:27.801431 2921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:27.802408 2921 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:27.804068 2921 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:27.804144 2921 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:27.804316 2921 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:27.804376 2921 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:27.841233 2921 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44127
I20260809 07:54:27.841275 3178 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44127 every 8 connection(s)
I20260809 07:54:27.842844 2921 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:27.849653 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 2921
I20260809 07:54:27.886703 3182 heartbeater.cc:344] Connected to a master server at 127.26.37.62:32907
I20260809 07:54:27.887379 3182 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:27.888401 3181 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36543
I20260809 07:54:27.888564 3181 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:27.888873 3182 heartbeater.cc:507] Master 127.26.37.62:32907 requested a full tablet report, sending...
I20260809 07:54:27.888998 3181 heartbeater.cc:507] Master 127.26.37.60:36543 requested a full tablet report, sending...
I20260809 07:54:27.891028 2424 ts_manager.cc:194] Re-registered known tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:27.891397 2237 ts_manager.cc:194] Re-registered known tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:27.895208 3184 heartbeater.cc:344] Connected to a master server at 127.26.37.61:43021
I20260809 07:54:27.895373 3184 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:27.895856 3184 heartbeater.cc:507] Master 127.26.37.61:43021 requested a full tablet report, sending...
I20260809 07:54:27.897166 2334 ts_manager.cc:194] Re-registered known tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:27.898010 3180 heartbeater.cc:344] Connected to a master server at 127.26.37.59:32839
I20260809 07:54:27.898142 3180 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:27.898265 2334 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:54339
I20260809 07:54:27.898573 3180 heartbeater.cc:507] Master 127.26.37.59:32839 requested a full tablet report, sending...
I20260809 07:54:27.900424 2941 ts_manager.cc:194] Registered new tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:27.908382 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:27.916666 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:27.916899 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2195
I20260809 07:54:27.917891 2277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:28.026438 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2195
I20260809 07:54:28.058444 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:32907
--webserver_interface=127.26.37.62
--webserver_port=42621
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839 with env {}
I20260809 07:54:28.100282 2888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:28.100627 2888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:28.102387 2888 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295: attempting to promote NON_VOTER 5c5bd434943444f6be029c425b85c047 to VOTER
I20260809 07:54:28.103216 2888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } }
I20260809 07:54:28.105152 2956 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 1 LEARNER]: Refusing update from remote peer 2bb76722ceea4534b94f93192d4c2295: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:54:28.105965 3201 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:54:28.106132 2439 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 1 FOLLOWER]: Refusing update from remote peer 2bb76722ceea4534b94f93192d4c2295: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:54:28.107159 3201 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 07:54:28.109799 2312 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 -> Peer 3fe198c78a81422b9d5890e314b6cc6a (127.26.37.62:32907): Couldn't send request to peer 3fe198c78a81422b9d5890e314b6cc6a. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:32907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:54:28.111557 3205 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5c5bd434943444f6be029c425b85c047 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:28.112370 2956 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5c5bd434943444f6be029c425b85c047 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:28.115127 3203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:28.115427 3203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:28.115326 3205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:28.115558 3205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:28.115954 2439 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5c5bd434943444f6be029c425b85c047 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:28.114969 3201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:28.118916 3201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:28.120738 3211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:28.121058 3211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6448dc3cba744fe5a58f50957bedc339 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:28.121469 3210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:28.121735 3210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5bd434943444f6be029c425b85c047 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:54:28.434296 3200 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:28.434635 3200 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:28.434765 3200 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:28.443120 3200 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:28.443203 3200 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:28.443324 3200 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:28.443406 3200 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:28.453413 3200 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=42621
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:28.455063 3200 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:28.456875 3200 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:28.465238 3222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:28.465359 3219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:28.465327 3218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:28.466500 3200 server_base.cc:1061] running on GCE node
I20260809 07:54:28.467281 3200 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:28.467479 3200 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:28.467578 3200 hybrid_clock.cc:648] HybridClock initialized: now 1786262068467555 us; error 0 us; skew 500 ppm
I20260809 07:54:28.467929 3200 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:28.470592 3200 webserver.cc:533] Webserver started at http://127.26.37.62:42621/ using document root <none> and password file <none>
I20260809 07:54:28.471346 3200 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:28.471515 3200 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:28.476636 3200 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:54:28.479435 3227 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:28.480741 3200 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:54:28.480922 3200 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "3fe198c78a81422b9d5890e314b6cc6a"
format_stamp: "Formatted at 2026-08-09 07:54:24 on dist-test-slave-bw88"
I20260809 07:54:28.481734 3200 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:28.504005 3200 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:28.504864 3200 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:28.505155 3200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:28.525998 3200 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:32907
I20260809 07:54:28.526041 3290 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:32907 every 8 connection(s)
I20260809 07:54:28.527693 3200 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:28.532236 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3200
I20260809 07:54:28.532665 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2921
I20260809 07:54:28.536190 3291 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:28.540602 3291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: Bootstrap starting.
I20260809 07:54:28.542858 3172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:28.559718 3291 log.cc:826] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:28.569173 3291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:28.569777 3291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: Bootstrap complete.
I20260809 07:54:28.577201 3291 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } }
I20260809 07:54:28.577976 3291 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe198c78a81422b9d5890e314b6cc6a, State: Initialized, Role: FOLLOWER
I20260809 07:54:28.578852 3291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } }
I20260809 07:54:28.579775 3300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: true } } }
I20260809 07:54:28.580083 3300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:28.580916 3291 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:28.591964 3314 catalog_manager.cc:1295] Loaded cluster ID: 3737cb6c87e04f83a6af2f245644c995
I20260809 07:54:28.592147 3314 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: loading cluster ID for follower catalog manager: success
I20260809 07:54:28.596549 3314 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: acquiring CA information for follower catalog manager: success
I20260809 07:54:28.599239 3314 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:28.646896 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2921
I20260809 07:54:28.674449 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44127
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--webserver_port=39489
--webserver_interface=127.26.37.1
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839 with env {}
I20260809 07:54:29.032282 3256 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5c5bd434943444f6be029c425b85c047 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:29.038256 3328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:29.038677 3328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:29.039146 3300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb76722ceea4534b94f93192d4c2295. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:29.039311 3300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe198c78a81422b9d5890e314b6cc6a [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:54:29.045691 3322 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:29.046002 3322 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:29.046135 3322 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:29.054456 3322 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:29.054656 3322 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:29.064487 3322 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44127
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39489
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:29.066223 3322 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:29.068094 3322 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:29.078342 3331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:29.078402 3334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:29.079064 3332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:29.079732 3322 server_base.cc:1061] running on GCE node
I20260809 07:54:29.080310 3322 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:29.080484 3322 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:29.080587 3322 hybrid_clock.cc:648] HybridClock initialized: now 1786262069080548 us; error 0 us; skew 500 ppm
I20260809 07:54:29.080888 3322 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:29.083177 3322 webserver.cc:533] Webserver started at http://127.26.37.1:39489/ using document root <none> and password file <none>
I20260809 07:54:29.083881 3322 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:29.083966 3322 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:29.097514 3322 fs_manager.cc:714] Time spent opening directory manager: real 0.011s user 0.007s sys 0.004s
I20260809 07:54:29.101225 3341 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:29.102700 3322 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:29.102862 3322 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "8064ba99193a4fb4957ac68f458a4983"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:29.103579 3322 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:29.120534 3322 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:29.121508 3322 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:29.121799 3322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:29.122743 3322 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:29.124357 3322 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:29.124430 3322 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:29.124619 3322 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:29.124699 3322 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:29.159230 3322 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44127
I20260809 07:54:29.159268 3489 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44127 every 8 connection(s)
I20260809 07:54:29.160782 3322 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:29.165042 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3322
I20260809 07:54:29.165509 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3322
I20260809 07:54:29.183300 3492 heartbeater.cc:344] Connected to a master server at 127.26.37.62:32907
I20260809 07:54:29.183782 3492 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:29.184995 3492 heartbeater.cc:507] Master 127.26.37.62:32907 requested a full tablet report, sending...
I20260809 07:54:29.188042 3246 ts_manager.cc:194] Registered new tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:29.189277 3490 heartbeater.cc:344] Connected to a master server at 127.26.37.59:32839
I20260809 07:54:29.189450 3490 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:29.189967 3490 heartbeater.cc:507] Master 127.26.37.59:32839 requested a full tablet report, sending...
I20260809 07:54:29.191272 3491 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36543
I20260809 07:54:29.191360 2941 ts_manager.cc:194] Re-registered known tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:29.191431 3491 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:29.191972 3485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:29.274912 3491 heartbeater.cc:507] Master 127.26.37.60:36543 requested a full tablet report, sending...
I20260809 07:54:29.276110 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3322
I20260809 07:54:29.276515 2424 ts_manager.cc:194] Re-registered known tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
W20260809 07:54:29.304653 2407 connection.cc:573] server connection from 127.26.37.1:48317 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:54:29.305457 3506 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:29.178870 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:55247 (local address 127.26.37.61:43021)
0809 07:54:29.193141 (+ 14271us) server_negotiation.cc:207] Beginning negotiation
0809 07:54:29.193150 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:54:29.193171 (+ 21us) server_negotiation.cc:408] Connection header received
0809 07:54:29.193231 (+ 60us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:54:29.193239 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:54:29.193317 (+ 78us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:54:29.193455 (+ 138us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:54:29.282565 (+ 89110us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:54:29.283586 (+ 1021us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:54:29.304453 (+ 20867us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:55247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":14150,"thread_start_us":14717,"threads_started":1}
I20260809 07:54:29.305794 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44127
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--webserver_port=39489
--webserver_interface=127.26.37.1
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839
--fault_before_start=1.0 with env {}
I20260809 07:54:29.475416 3201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bb76722ceea4534b94f93192d4c2295" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3fe198c78a81422b9d5890e314b6cc6a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 32907 } } peers { permanent_uuid: "2bb76722ceea4534b94f93192d4c2295" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43021 } } peers { permanent_uuid: "6448dc3cba744fe5a58f50957bedc339" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36543 } } peers { permanent_uuid: "5c5bd434943444f6be029c425b85c047" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 32839 } attrs { promote: false } } }
I20260809 07:54:29.475754 3201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb76722ceea4534b94f93192d4c2295 [sys.catalog]: This master's current role is: LEADER
W20260809 07:54:29.673911 3510 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:29.674213 3510 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:29.674292 3510 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:54:29.674350 3510 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:29.682761 3510 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:29.682948 3510 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:29.693046 3510 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44127
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39489
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:29.694794 3510 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:29.696688 3510 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:29.706804 3524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:29.707612 3525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:29.709555 3530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:29.710057 3510 server_base.cc:1061] running on GCE node
I20260809 07:54:29.710672 3510 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:29.710852 3510 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:29.710973 3510 hybrid_clock.cc:648] HybridClock initialized: now 1786262069710927 us; error 0 us; skew 500 ppm
I20260809 07:54:29.711283 3510 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:29.713485 3510 webserver.cc:533] Webserver started at http://127.26.37.1:39489/ using document root <none> and password file <none>
I20260809 07:54:29.714284 3510 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:29.714437 3510 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:29.720220 3510 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:54:29.723160 3537 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:29.724543 3510 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:54:29.724720 3510 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "8064ba99193a4fb4957ac68f458a4983"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:29.725531 3510 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:29.739792 3510 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:29.740679 3510 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:29.740972 3510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:29.742022 3510 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:29.743733 3510 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:29.743870 3510 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:29.743988 3510 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:29.744077 3510 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:54:29.744525 3510 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:54:29.744638 3510 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:54:29.744729 3510 fault_injection.cc:59] injecting fault for kudu (pid 3510): FLAGS_fault_before_start (process will exit)
I20260809 07:54:29.772099 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:44127
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543
--webserver_port=39489
--webserver_interface=127.26.37.1
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839 with env {}
W20260809 07:54:30.142601 3546 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:30.142907 3546 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:30.142992 3546 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:30.151382 3546 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:30.151557 3546 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:30.161725 3546 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:44127
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=39489
--tserver_master_addrs=127.26.37.62:32907,127.26.37.61:43021,127.26.37.60:36543,127.26.37.59:32839
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:30.163417 3546 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:30.165328 3546 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:30.175081 3552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:30.175051 3551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:30.175835 3554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:30.176338 3546 server_base.cc:1061] running on GCE node
I20260809 07:54:30.176928 3546 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:54:30.177067 3546 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:54:30.177134 3546 hybrid_clock.cc:648] HybridClock initialized: now 1786262070177115 us; error 0 us; skew 500 ppm
I20260809 07:54:30.177481 3546 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:30.179611 3546 webserver.cc:533] Webserver started at http://127.26.37.1:39489/ using document root <none> and password file <none>
I20260809 07:54:30.180322 3546 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:30.180420 3546 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:30.185889 3546 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:54:30.188844 3562 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:30.190196 3546 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:30.190342 3546 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "8064ba99193a4fb4957ac68f458a4983"
format_stamp: "Formatted at 2026-08-09 07:54:26 on dist-test-slave-bw88"
I20260809 07:54:30.191027 3546 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:30.213438 3546 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:30.214376 3546 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:30.214620 3546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:30.215561 3546 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:30.217330 3546 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:30.217453 3546 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 07:54:30.217598 3546 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:30.217667 3546 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:30.254711 3546 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44127
I20260809 07:54:30.254750 3710 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44127 every 8 connection(s)
I20260809 07:54:30.256714 3546 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:30.258209 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3546
I20260809 07:54:30.258622 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3546
I20260809 07:54:30.297087 3713 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36543
I20260809 07:54:30.297549 3713 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:30.298028 3717 heartbeater.cc:344] Connected to a master server at 127.26.37.61:43021
I20260809 07:54:30.298203 3717 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:30.298590 3706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:30.386909 3711 heartbeater.cc:344] Connected to a master server at 127.26.37.59:32839
I20260809 07:54:30.387385 3717 heartbeater.cc:507] Master 127.26.37.61:43021 requested a full tablet report, sending...
I20260809 07:54:30.388118 3713 heartbeater.cc:507] Master 127.26.37.60:36543 requested a full tablet report, sending...
I20260809 07:54:30.388414 3716 heartbeater.cc:344] Connected to a master server at 127.26.37.62:32907
I20260809 07:54:30.388643 3716 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:30.389384 3711 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:30.391043 3716 heartbeater.cc:507] Master 127.26.37.62:32907 requested a full tablet report, sending...
I20260809 07:54:30.391135 3711 heartbeater.cc:507] Master 127.26.37.59:32839 requested a full tablet report, sending...
I20260809 07:54:30.392074 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3546
I20260809 07:54:30.392710 2941 ts_manager.cc:194] Re-registered known tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:30.392802 3246 ts_manager.cc:194] Re-registered known tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
I20260809 07:54:30.393662 2424 ts_manager.cc:194] Re-registered known tserver with Master: 8064ba99193a4fb4957ac68f458a4983 (127.26.37.1:44127)
W20260809 07:54:30.418969 2919 connection.cc:573] server connection from 127.26.37.1:56221 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:54:30.418982 3229 connection.cc:573] server connection from 127.26.37.1:37417 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:30.419371 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3200
W20260809 07:54:30.419375 2409 connection.cc:573] server connection from 127.26.37.1:40647 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:30.426216 3284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:30.535776 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3200
I20260809 07:54:30.564074 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2292
W20260809 07:54:30.564709 3735 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:54:30.448824 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.26.37.62:32907 (local address 127.0.0.1:43400)
0809 07:54:30.450223 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0809 07:54:30.450240 (+ 17us) client_negotiation.cc:175] Beginning negotiation
0809 07:54:30.450367 (+ 127us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0809 07:54:30.535472 (+ 85105us) client_negotiation.cc:281] Received NEGOTIATE NegotiatePB response
0809 07:54:30.535478 (+ 6us) client_negotiation.cc:404] Received NEGOTIATE response from server
0809 07:54:30.535523 (+ 45us) client_negotiation.cc:190] Negotiated authn=CERTIFICATE
0809 07:54:30.536848 (+ 1325us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0809 07:54:30.536865 (+ 17us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:54:30.563505 (+ 26640us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.37.62:32907: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1262,"thread_start_us":1140,"threads_started":1}
I20260809 07:54:30.565078 2381 master_runner.cc:187] Error getting master registration for 127.26.37.62:32907: Network error: Client connection negotiation failed: client connection to 127.26.37.62:32907: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260809 07:54:30.565290 2381 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:54:30.565526 2370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:30.679777 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2292
I20260809 07:54:30.706988 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2381
I20260809 07:54:30.708243 2467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:30.818670 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2381
I20260809 07:54:30.845233 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 2752
I20260809 07:54:30.846537 2986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:30.958317 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 2752
[ OK ] ExternalMiniClusterTest.TestAddMaster/0 (7106 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/1
2026-08-09T07:54:31Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:31Z Disabled control of system clock
I20260809 07:54:31.052934 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41169
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257 with env {}
W20260809 07:54:31.414351 3746 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:31.414667 3746 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:31.414740 3746 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:31.422897 3746 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:31.423000 3746 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:31.423058 3746 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:31.423105 3746 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:31.432564 3746 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:31.434230 3746 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:31.436036 3746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:31.443773 3752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:31.443786 3751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:31.445109 3754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:31.445775 3746 server_base.cc:1061] running on GCE node
I20260809 07:54:31.446563 3746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:31.447980 3746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:31.449211 3746 hybrid_clock.cc:648] HybridClock initialized: now 1786262071449164 us; error 40 us; skew 500 ppm
I20260809 07:54:31.449586 3746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:31.451776 3746 webserver.cc:533] Webserver started at http://127.26.37.62:45307/ using document root <none> and password file <none>
I20260809 07:54:31.452518 3746 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:31.452656 3746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:31.453027 3746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:31.455442 3746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "8cd6890316e440c7b2d9af6a446e0e1a"
format_stamp: "Formatted at 2026-08-09 07:54:31 on dist-test-slave-bw88"
I20260809 07:54:31.456202 3746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "8cd6890316e440c7b2d9af6a446e0e1a"
format_stamp: "Formatted at 2026-08-09 07:54:31 on dist-test-slave-bw88"
I20260809 07:54:31.461560 3746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260809 07:54:31.465041 3762 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:31.466430 3746 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:54:31.466636 3746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "8cd6890316e440c7b2d9af6a446e0e1a"
format_stamp: "Formatted at 2026-08-09 07:54:31 on dist-test-slave-bw88"
I20260809 07:54:31.466866 3746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:31.503518 3746 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:31.504521 3746 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:31.504838 3746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:31.525943 3746 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41169
I20260809 07:54:31.525993 3832 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41169 every 8 connection(s)
I20260809 07:54:31.527585 3746 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:31.532042 3833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:31.535053 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3746
I20260809 07:54:31.535256 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:54:31.536290 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:32783
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257 with env {}
I20260809 07:54:31.537895 3833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:31.568843 3833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:31.570335 3764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:32783: connect: Connection refused (error 111)
W20260809 07:54:31.572160 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:32783: Network error: Client connection negotiation failed: client connection to 127.26.37.61:32783: connect: Connection refused (error 111)
I20260809 07:54:31.621547 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } attempt: 1
W20260809 07:54:31.624061 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:32783: Network error: Client connection negotiation failed: client connection to 127.26.37.61:32783: connect: Connection refused (error 111)
I20260809 07:54:31.692364 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } attempt: 2
W20260809 07:54:31.694758 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:32783: Network error: Client connection negotiation failed: client connection to 127.26.37.61:32783: connect: Connection refused (error 111)
I20260809 07:54:31.786069 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } attempt: 3
W20260809 07:54:31.788443 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:32783: Network error: Client connection negotiation failed: client connection to 127.26.37.61:32783: connect: Connection refused (error 111)
W20260809 07:54:31.903208 3844 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:31.903568 3844 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:31.903739 3844 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:31.911834 3844 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:31.911935 3844 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:31.912037 3844 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:31.912123 3844 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:31.921901 3844 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:32783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:31.923650 3844 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:31.925542 3844 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:54:31.931739 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } attempt: 4
W20260809 07:54:31.934371 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:32783: Network error: Client connection negotiation failed: client connection to 127.26.37.61:32783: connect: Connection refused (error 111)
W20260809 07:54:31.935043 3853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:31.935704 3852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:31.936352 3855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:31.937592 3844 server_base.cc:1061] running on GCE node
I20260809 07:54:31.938359 3844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:31.939826 3844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:31.941061 3844 hybrid_clock.cc:648] HybridClock initialized: now 1786262071941003 us; error 51 us; skew 500 ppm
I20260809 07:54:31.941481 3844 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:31.943673 3844 webserver.cc:533] Webserver started at http://127.26.37.61:41801/ using document root <none> and password file <none>
I20260809 07:54:31.944407 3844 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:31.944551 3844 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:31.944918 3844 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:31.947386 3844 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "dafd435bf9f441c2a7a25c35f1d23c1c"
format_stamp: "Formatted at 2026-08-09 07:54:31 on dist-test-slave-bw88"
I20260809 07:54:31.948177 3844 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "dafd435bf9f441c2a7a25c35f1d23c1c"
format_stamp: "Formatted at 2026-08-09 07:54:31 on dist-test-slave-bw88"
I20260809 07:54:31.953655 3844 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 07:54:31.956961 3862 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:31.958316 3844 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:31.958552 3844 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "dafd435bf9f441c2a7a25c35f1d23c1c"
format_stamp: "Formatted at 2026-08-09 07:54:31 on dist-test-slave-bw88"
I20260809 07:54:31.958799 3844 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:31.982787 3844 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:31.983639 3844 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:31.983932 3844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:32.003932 3844 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:32783
I20260809 07:54:32.003954 3932 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:32783 every 8 connection(s)
I20260809 07:54:32.005607 3844 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:54:32.009814 3933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:32.011415 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3844
I20260809 07:54:32.011644 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:54:32.012642 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:36257
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257 with env {}
I20260809 07:54:32.015583 3933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:32.028367 3933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:32.038218 3933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:32.039348 3864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
W20260809 07:54:32.041046 3933 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36257: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
I20260809 07:54:32.090438 3933 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } attempt: 1
W20260809 07:54:32.092849 3933 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36257: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
I20260809 07:54:32.161109 3933 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } attempt: 2
W20260809 07:54:32.163445 3933 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36257: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
I20260809 07:54:32.233683 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } attempt: 5
I20260809 07:54:32.240375 3833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:32.242434 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36257: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
I20260809 07:54:32.254761 3933 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } attempt: 3
W20260809 07:54:32.256907 3933 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36257: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
I20260809 07:54:32.293684 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } attempt: 1
W20260809 07:54:32.295734 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36257: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
I20260809 07:54:32.364015 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } attempt: 2
W20260809 07:54:32.366166 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36257: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
W20260809 07:54:32.371475 3937 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:32.371769 3937 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:32.371856 3937 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:32.379943 3937 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:32.380016 3937 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:32.380066 3937 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:32.380129 3937 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:32.389815 3937 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:36257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:32.391386 3937 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:32.393203 3937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:54:32.400152 3933 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } attempt: 4
W20260809 07:54:32.402208 3946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:32.403383 3933 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:36257: Network error: Client connection negotiation failed: client connection to 127.26.37.60:36257: connect: Connection refused (error 111)
W20260809 07:54:32.403522 3947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:32.404434 3953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:32.404543 3937 server_base.cc:1061] running on GCE node
I20260809 07:54:32.405428 3937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:32.406801 3937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:32.408105 3937 hybrid_clock.cc:648] HybridClock initialized: now 1786262072408062 us; error 32 us; skew 500 ppm
I20260809 07:54:32.408430 3937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:32.410676 3937 webserver.cc:533] Webserver started at http://127.26.37.60:42903/ using document root <none> and password file <none>
I20260809 07:54:32.411388 3937 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:32.411480 3937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:32.411794 3937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:32.414168 3937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "73489c7d312049f5adac35ca25ef23c6"
format_stamp: "Formatted at 2026-08-09 07:54:32 on dist-test-slave-bw88"
I20260809 07:54:32.414928 3937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "73489c7d312049f5adac35ca25ef23c6"
format_stamp: "Formatted at 2026-08-09 07:54:32 on dist-test-slave-bw88"
I20260809 07:54:32.420182 3937 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:54:32.423709 3959 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:32.425024 3937 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:54:32.425163 3937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "73489c7d312049f5adac35ca25ef23c6"
format_stamp: "Formatted at 2026-08-09 07:54:32 on dist-test-slave-bw88"
I20260809 07:54:32.425391 3937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:32.441033 3937 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:32.441952 3937 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:32.442185 3937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:32.465204 3937 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:36257
I20260809 07:54:32.465224 4032 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:36257 every 8 connection(s)
I20260809 07:54:32.466832 3937 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:54:32.470217 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 3937
I20260809 07:54:32.470434 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:54:32.472119 4033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:32.472446 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } attempt: 3
I20260809 07:54:32.482698 4033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:32.496479 4033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:32.500236 3833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Bootstrap starting.
I20260809 07:54:32.502662 4033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:32.505113 3833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:32.506875 3833 log.cc:826] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:32.509824 3833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: No bootstrap required, opened a new log
I20260809 07:54:32.512316 4033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: Bootstrap starting.
I20260809 07:54:32.515702 4033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:32.516492 3833 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.516755 3833 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:32.516769 4033 log.cc:826] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:32.516908 3833 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd6890316e440c7b2d9af6a446e0e1a, State: Initialized, Role: FOLLOWER
I20260809 07:54:32.517642 3833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.518705 4044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:32.519004 4044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:32.519498 4033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: No bootstrap required, opened a new log
I20260809 07:54:32.520213 3833 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:32.525525 4033 raft_consensus.cc:359] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.525949 4033 raft_consensus.cc:385] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:32.526129 4033 raft_consensus.cc:740] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73489c7d312049f5adac35ca25ef23c6, State: Initialized, Role: FOLLOWER
I20260809 07:54:32.526952 4033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.534125 4058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:32.535996 4058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:32.535871 4033 sys_catalog.cc:565] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:54:32.543974 4057 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:32.544137 4057 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:32.535619 3746 master_runner.cc:187] Error getting master registration for 127.26.37.61:32783: OK, instance_id { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" instance_seqno: 1786262071998591 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:54:32.549209 3746 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:54:32.567299 4078 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:32.567488 4078 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:32.559609 3937 master_runner.cc:187] Error getting master registration for 127.26.37.61:32783: OK, instance_id { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" instance_seqno: 1786262071998591 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:54:32.569576 3937 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:54:32.694291 3966 tablet.cc:2406] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:54:32.702643 3933 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } attempt: 5
I20260809 07:54:32.716899 3933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: Bootstrap starting.
I20260809 07:54:32.721835 3933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:32.723385 3933 log.cc:826] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:32.726819 3933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: No bootstrap required, opened a new log
I20260809 07:54:32.735893 3933 raft_consensus.cc:359] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.736268 3933 raft_consensus.cc:385] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:32.736394 3933 raft_consensus.cc:740] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd435bf9f441c2a7a25c35f1d23c1c, State: Initialized, Role: FOLLOWER
I20260809 07:54:32.737169 3933 consensus_queue.cc:260] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.738350 4083 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:32.738734 4083 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:32.739820 3933 sys_catalog.cc:565] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:54:32.757771 3770 tablet.cc:2406] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:54:32.762372 3844 master_runner.cc:264] No leader master found from master dafd435bf9f441c2a7a25c35f1d23c1c
I20260809 07:54:32.762516 3844 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:54:32.765486 4098 catalog_manager.cc:1594] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:32.765661 4098 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:32.876521 4044 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:54:32.876830 4044 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.881590 3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73489c7d312049f5adac35ca25ef23c6" is_pre_election: true
I20260809 07:54:32.882215 3997 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cd6890316e440c7b2d9af6a446e0e1a in term 0.
I20260809 07:54:32.883152 3764 leader_election.cc:304] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73489c7d312049f5adac35ca25ef23c6, 8cd6890316e440c7b2d9af6a446e0e1a; no voters:
I20260809 07:54:32.884443 3897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" is_pre_election: true
I20260809 07:54:32.884992 3897 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cd6890316e440c7b2d9af6a446e0e1a in term 0.
I20260809 07:54:32.878850 4044 leader_election.cc:290] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dafd435bf9f441c2a7a25c35f1d23c1c (127.26.37.61:32783), 73489c7d312049f5adac35ca25ef23c6 (127.26.37.60:36257)
I20260809 07:54:32.885823 4044 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:54:32.885990 4044 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:54:32.886127 4044 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:32.891570 4044 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.893074 4044 leader_election.cc:290] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [CANDIDATE]: Term 1 election: Requested vote from peers dafd435bf9f441c2a7a25c35f1d23c1c (127.26.37.61:32783), 73489c7d312049f5adac35ca25ef23c6 (127.26.37.60:36257)
I20260809 07:54:32.904812 3897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c"
I20260809 07:54:32.905084 3897 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:32.908730 3897 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cd6890316e440c7b2d9af6a446e0e1a in term 1.
I20260809 07:54:32.909950 3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73489c7d312049f5adac35ca25ef23c6"
I20260809 07:54:32.910210 3997 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:32.913980 3997 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cd6890316e440c7b2d9af6a446e0e1a in term 1.
I20260809 07:54:32.915151 3764 leader_election.cc:304] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cd6890316e440c7b2d9af6a446e0e1a, dafd435bf9f441c2a7a25c35f1d23c1c; no voters:
I20260809 07:54:32.915577 4044 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:54:32.915799 4044 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 1 LEADER]: Becoming Leader. State: Replica: 8cd6890316e440c7b2d9af6a446e0e1a, State: Running, Role: LEADER
I20260809 07:54:32.916455 4044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } }
I20260809 07:54:32.928782 4102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cd6890316e440c7b2d9af6a446e0e1a. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:32.929292 4102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:32.936478 4105 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:54:32.939079 4105 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:54:32.954906 3997 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Refusing update from remote peer 8cd6890316e440c7b2d9af6a446e0e1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:54:32.956086 4102 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:54:32.968117 3897 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Refusing update from remote peer 8cd6890316e440c7b2d9af6a446e0e1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:54:32.969235 4102 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:54:32.986635 3869 tablet.cc:2406] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:54:33.015990 4083 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cd6890316e440c7b2d9af6a446e0e1a. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:33.016304 4083 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:33.020323 4106 mvcc.cc:204] Tried to move back new op lower bound from 7316529450812583936 to 7316529450671640576. Current Snapshot: MvccSnapshot[applied={T|T < 7316529450812583936}]
I20260809 07:54:33.027793 4044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:33.028100 4044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:33.030244 4105 catalog_manager.cc:1383] Generated new cluster ID: 3eb3e5624e7c4e99a09250562e9872bd
I20260809 07:54:33.030393 4105 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:54:33.041501 4083 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:33.041776 4083 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:33.043238 4058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cd6890316e440c7b2d9af6a446e0e1a. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:33.043450 4058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:33.050146 4115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:33.050462 4115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:33.053663 4107 mvcc.cc:204] Tried to move back new op lower bound from 7316529450812583936 to 7316529450671640576. Current Snapshot: MvccSnapshot[applied={T|T < 7316529450812583936}]
I20260809 07:54:33.058519 4058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } }
I20260809 07:54:33.058720 4058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:33.091447 4105 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:54:33.096357 4105 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:54:33.121219 4105 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Generated new TSK 0
I20260809 07:54:33.122375 4105 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:54:33.153524 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:54:33.540894 4118 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:33.541236 4118 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:33.541414 4118 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:33.549918 4118 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:33.550122 4118 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:33.560109 4118 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:33.561830 4118 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:33.563748 4118 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:54:33.569763 4078 catalog_manager.cc:1295] Loaded cluster ID: 3eb3e5624e7c4e99a09250562e9872bd
I20260809 07:54:33.569878 4078 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: loading cluster ID for follower catalog manager: success
I20260809 07:54:33.573689 4078 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: acquiring CA information for follower catalog manager: success
W20260809 07:54:33.574406 4127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:33.574586 4125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:33.575857 4078 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:54:33.577610 4131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:33.578234 4118 server_base.cc:1061] running on GCE node
I20260809 07:54:33.578907 4118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:33.579747 4118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:33.581022 4118 hybrid_clock.cc:648] HybridClock initialized: now 1786262073580970 us; error 32 us; skew 500 ppm
I20260809 07:54:33.581323 4118 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:33.583528 4118 webserver.cc:533] Webserver started at http://127.26.37.1:34429/ using document root <none> and password file <none>
I20260809 07:54:33.584242 4118 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:33.584331 4118 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:33.584724 4118 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:33.587124 4118 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "2fad6d19572647f28e6c25d688eee5db"
format_stamp: "Formatted at 2026-08-09 07:54:33 on dist-test-slave-bw88"
I20260809 07:54:33.587975 4118 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "2fad6d19572647f28e6c25d688eee5db"
format_stamp: "Formatted at 2026-08-09 07:54:33 on dist-test-slave-bw88"
I20260809 07:54:33.593250 4118 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:54:33.596761 4142 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:33.598165 4118 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:33.598305 4118 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2fad6d19572647f28e6c25d688eee5db"
format_stamp: "Formatted at 2026-08-09 07:54:33 on dist-test-slave-bw88"
I20260809 07:54:33.598506 4118 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:33.614611 4118 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:33.615556 4118 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:33.615790 4118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:33.616776 4118 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:33.618609 4118 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:33.618717 4118 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:33.618798 4118 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:33.618852 4118 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:33.659538 4118 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34047
I20260809 07:54:33.659598 4292 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34047 every 8 connection(s)
I20260809 07:54:33.661067 4118 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:33.666318 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4118
I20260809 07:54:33.666580 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:54:33.692087 4294 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41169
I20260809 07:54:33.692380 4295 heartbeater.cc:344] Connected to a master server at 127.26.37.61:32783
I20260809 07:54:33.692590 4295 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:33.692798 4293 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36257
I20260809 07:54:33.692914 4293 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:33.692590 4294 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:33.693888 4294 heartbeater.cc:507] Master 127.26.37.62:41169 requested a full tablet report, sending...
I20260809 07:54:33.693946 4295 heartbeater.cc:507] Master 127.26.37.61:32783 requested a full tablet report, sending...
I20260809 07:54:33.694250 4293 heartbeater.cc:507] Master 127.26.37.60:36257 requested a full tablet report, sending...
I20260809 07:54:33.696260 3882 ts_manager.cc:194] Registered new tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:33.696318 3782 ts_manager.cc:194] Registered new tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:33.696355 3981 ts_manager.cc:194] Registered new tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:33.698123 3782 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:59499
I20260809 07:54:33.710568 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:33.712858 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:44489
--webserver_interface=127.26.37.59
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489 with env {}
I20260809 07:54:33.767557 4098 catalog_manager.cc:1295] Loaded cluster ID: 3eb3e5624e7c4e99a09250562e9872bd
I20260809 07:54:33.767660 4098 catalog_manager.cc:1588] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: loading cluster ID for follower catalog manager: success
I20260809 07:54:33.770308 4098 catalog_manager.cc:1610] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: acquiring CA information for follower catalog manager: success
I20260809 07:54:33.772305 4098 catalog_manager.cc:1638] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:54:34.074237 4308 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:34.074672 4308 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:34.074805 4308 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:34.083084 4308 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:34.083220 4308 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:34.083345 4308 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:34.083428 4308 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:34.093721 4308 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:44489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:34.095412 4308 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:34.097370 4308 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:34.106222 4315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:34.106174 4314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:34.106925 4318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:34.107604 4308 server_base.cc:1061] running on GCE node
I20260809 07:54:34.108362 4308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:34.109843 4308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:34.111081 4308 hybrid_clock.cc:648] HybridClock initialized: now 1786262074111042 us; error 49 us; skew 500 ppm
I20260809 07:54:34.111444 4308 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:34.113747 4308 webserver.cc:533] Webserver started at http://127.26.37.59:37641/ using document root <none> and password file <none>
I20260809 07:54:34.114540 4308 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:34.114682 4308 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:34.115070 4308 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:34.117874 4308 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "82c49856cb164f5fa90bdb87b4e08cdf"
format_stamp: "Formatted at 2026-08-09 07:54:34 on dist-test-slave-bw88"
I20260809 07:54:34.118963 4308 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "82c49856cb164f5fa90bdb87b4e08cdf"
format_stamp: "Formatted at 2026-08-09 07:54:34 on dist-test-slave-bw88"
I20260809 07:54:34.124861 4308 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 07:54:34.128356 4325 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:34.129874 4308 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:34.130061 4308 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "82c49856cb164f5fa90bdb87b4e08cdf"
format_stamp: "Formatted at 2026-08-09 07:54:34 on dist-test-slave-bw88"
I20260809 07:54:34.130311 4308 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:34.147498 4308 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:34.148335 4308 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:34.148604 4308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:34.169271 4308 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:44489
I20260809 07:54:34.169279 4389 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:44489 every 8 connection(s)
I20260809 07:54:34.170912 4308 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:54:34.175462 4390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:34.178216 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4308
I20260809 07:54:34.182770 4390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:34.193418 4390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:34.199708 4390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:34.205500 4390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:34.214193 4390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Bootstrap starting.
I20260809 07:54:34.217229 4390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:34.218272 4390 log.cc:826] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:34.220844 4390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: No bootstrap required, opened a new log
I20260809 07:54:34.226222 4390 raft_consensus.cc:359] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } }
I20260809 07:54:34.226496 4390 raft_consensus.cc:385] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:34.226583 4390 raft_consensus.cc:740] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c49856cb164f5fa90bdb87b4e08cdf, State: Initialized, Role: FOLLOWER
I20260809 07:54:34.227188 4390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } }
I20260809 07:54:34.228111 4401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } } }
I20260809 07:54:34.228368 4401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:34.229110 4390 sys_catalog.cc:565] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:54:34.238328 4418 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:34.238480 4418 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:34.245590 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4118
I20260809 07:54:34.246482 4308 master_runner.cc:429] Detected that this master 82c49856cb164f5fa90bdb87b4e08cdf is joining an existing cluster
I20260809 07:54:34.246591 4308 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:54:34.247820 4308 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:54:34.248188 4308 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:54:34.248423 4308 tablet_replica.cc:333] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: stopping tablet replica
I20260809 07:54:34.250392 4308 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:44489
I20260809 07:54:34.254792 4287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:34.306180 3782 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:44489: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 44489
}
attrs {
promote: true
}
}
dest_uuid: "8cd6890316e440c7b2d9af6a446e0e1a"
cas_config_opid_index: -1
I20260809 07:54:34.307986 3782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:34.311704 3894 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Refusing update from remote peer 8cd6890316e440c7b2d9af6a446e0e1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260809 07:54:34.311875 3763 consensus_peers.cc:597] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a -> Peer 82c49856cb164f5fa90bdb87b4e08cdf (127.26.37.59:44489): Couldn't send request to peer 82c49856cb164f5fa90bdb87b4e08cdf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:54:34.312311 3997 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Refusing update from remote peer 8cd6890316e440c7b2d9af6a446e0e1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:54:34.312975 4116 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:54:34.313555 4116 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:54:34.320503 4116 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 82c49856cb164f5fa90bdb87b4e08cdf (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.326506 3997 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 82c49856cb164f5fa90bdb87b4e08cdf (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.332342 4116 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:44489
I20260809 07:54:34.333245 4308 master.cc:562] Master@127.26.37.59:44489 shutting down...
I20260809 07:54:34.334766 4438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.335139 4438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:34.345911 4116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.346200 4116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:34.346745 3894 raft_consensus.cc:2955] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 82c49856cb164f5fa90bdb87b4e08cdf (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.349783 4441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.350147 4441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:34.350706 4308 master.cc:584] Master@127.26.37.59:44489 shutdown complete.
I20260809 07:54:34.350925 4308 master_runner.cc:306] Clearing existing system tablet
I20260809 07:54:34.351455 4308 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:34.351653 4308 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:34.353819 4444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.354203 4444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:34.357610 4448 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.357975 4448 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:34.358014 4308 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 07:54:34.359699 4452 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:34.360060 4308 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:34.360302 4308 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 07:54:34.360404 4308 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "82c49856cb164f5fa90bdb87b4e08cdf"
format_stamp: "Formatted at 2026-08-09 07:54:34 on dist-test-slave-bw88"
I20260809 07:54:34.364734 4308 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:54:34.371534 4308 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:54:34.371708 4308 log.cc:1199] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:54:34.372288 4308 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Deleting consensus metadata
I20260809 07:54:34.372557 4308 master_runner.cc:316] Copying system tablet from 127.26.37.62:41169
I20260809 07:54:34.375334 4308 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.62:41169
I20260809 07:54:34.385001 3812 tablet_copy_service.cc:140] P 8cd6890316e440c7b2d9af6a446e0e1a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 82c49856cb164f5fa90bdb87b4e08cdf ({username='slave'} at 127.0.0.1:54798)
I20260809 07:54:34.385308 3812 tablet_copy_service.cc:161] P 8cd6890316e440c7b2d9af6a446e0e1a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 82c49856cb164f5fa90bdb87b4e08cdf at {username='slave'} at 127.0.0.1:54798: session id = 82c49856cb164f5fa90bdb87b4e08cdf-00000000000000000000000000000000
I20260809 07:54:34.389871 3812 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:54:34.392326 4308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:54:34.394783 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4118
I20260809 07:54:34.406363 4308 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: tablet copy: Starting download of 0 data blocks...
I20260809 07:54:34.406735 4308 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: tablet copy: Starting download of 1 WAL segments...
I20260809 07:54:34.409904 4308 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:54:34.414696 3812 tablet_copy_service.cc:342] P 8cd6890316e440c7b2d9af6a446e0e1a: Request end of tablet copy session 82c49856cb164f5fa90bdb87b4e08cdf-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:54798
I20260809 07:54:34.414986 3812 tablet_copy_service.cc:434] P 8cd6890316e440c7b2d9af6a446e0e1a: ending tablet copy session 82c49856cb164f5fa90bdb87b4e08cdf-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 82c49856cb164f5fa90bdb87b4e08cdf
W20260809 07:54:34.424322 4308 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260809 07:54:34.427413 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34047
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--webserver_port=34429
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489 with env {}
I20260809 07:54:34.428645 4308 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:34.431378 4464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:34.431838 4465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:34.439087 4468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:34.445796 4308 server_base.cc:1061] running on GCE node
I20260809 07:54:34.446158 4308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:34.446730 4308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:34.447955 4308 hybrid_clock.cc:648] HybridClock initialized: now 1786262074447880 us; error 35 us; skew 500 ppm
I20260809 07:54:34.448186 4308 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:34.456938 4308 webserver.cc:533] Webserver started at http://127.26.37.59:38865/ using document root <none> and password file <none>
I20260809 07:54:34.457309 4308 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:34.457506 4308 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:34.459923 4308 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 07:54:34.461766 4479 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:34.462261 4308 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 07:54:34.462404 4308 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "82c49856cb164f5fa90bdb87b4e08cdf"
format_stamp: "Formatted at 2026-08-09 07:54:34 on dist-test-slave-bw88"
I20260809 07:54:34.462589 4308 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:34.475265 4308 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:34.476236 4308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:34.514184 4308 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:44489
I20260809 07:54:34.514284 4544 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:44489 every 8 connection(s)
I20260809 07:54:34.515496 4308 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:54:34.520112 4545 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:34.523756 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Bootstrap starting.
I20260809 07:54:34.557662 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:34.558350 4545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: Bootstrap complete.
I20260809 07:54:34.559659 4545 raft_consensus.cc:359] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:34.559950 4545 raft_consensus.cc:740] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 82c49856cb164f5fa90bdb87b4e08cdf, State: Initialized, Role: LEARNER
I20260809 07:54:34.560288 4545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:34.561880 4550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.562145 4545 sys_catalog.cc:565] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:34.562222 4550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: This master's current role is: LEARNER
I20260809 07:54:34.573020 4573 catalog_manager.cc:1295] Loaded cluster ID: 3eb3e5624e7c4e99a09250562e9872bd
I20260809 07:54:34.573132 4573 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: loading cluster ID for follower catalog manager: success
I20260809 07:54:34.577067 4573 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: acquiring CA information for follower catalog manager: success
I20260809 07:54:34.579525 4573 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:34.718214 4514 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:54:34.719129 4550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cd6890316e440c7b2d9af6a446e0e1a. Latest consensus state: current_term: 1 leader_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:34.719432 4550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: This master's current role is: LEARNER
W20260809 07:54:34.841851 4472 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:34.842217 4472 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:34.842362 4472 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:34.851660 4472 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:34.851882 4472 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:34.862421 4472 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34047
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34429
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:34.864364 4472 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:34.866377 4472 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:34.876314 4582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:34.876363 4583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:34.877290 4585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:34.877923 4472 server_base.cc:1061] running on GCE node
I20260809 07:54:34.878563 4472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:34.879441 4472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:34.880652 4472 hybrid_clock.cc:648] HybridClock initialized: now 1786262074880605 us; error 32 us; skew 500 ppm
I20260809 07:54:34.880980 4472 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:34.883222 4472 webserver.cc:533] Webserver started at http://127.26.37.1:34429/ using document root <none> and password file <none>
I20260809 07:54:34.884003 4472 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:34.884160 4472 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:34.889624 4472 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:34.892379 4598 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:34.893797 4472 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:34.893970 4472 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2fad6d19572647f28e6c25d688eee5db"
format_stamp: "Formatted at 2026-08-09 07:54:33 on dist-test-slave-bw88"
I20260809 07:54:34.894678 4472 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:34.918382 4472 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:34.919317 4472 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:34.919644 4472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:34.920639 4472 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:34.922536 4472 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:34.922650 4472 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:34.922794 4472 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:34.922869 4472 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:34.958678 4472 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34047
I20260809 07:54:34.958727 4755 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34047 every 8 connection(s)
I20260809 07:54:34.960211 4472 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:34.961946 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4472
I20260809 07:54:35.006711 4756 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36257
I20260809 07:54:35.007205 4756 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:35.008483 4756 heartbeater.cc:507] Master 127.26.37.60:36257 requested a full tablet report, sending...
I20260809 07:54:35.008647 4758 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41169
I20260809 07:54:35.008801 4758 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:35.009361 4758 heartbeater.cc:507] Master 127.26.37.62:41169 requested a full tablet report, sending...
I20260809 07:54:35.008653 4766 heartbeater.cc:344] Connected to a master server at 127.26.37.61:32783
I20260809 07:54:35.009768 4766 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:35.010277 4766 heartbeater.cc:507] Master 127.26.37.61:32783 requested a full tablet report, sending...
I20260809 07:54:35.009639 4757 heartbeater.cc:344] Connected to a master server at 127.26.37.59:44489
I20260809 07:54:35.010558 4757 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:35.011075 4757 heartbeater.cc:507] Master 127.26.37.59:44489 requested a full tablet report, sending...
I20260809 07:54:35.011072 3981 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:35.011650 3882 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:35.011644 3782 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:35.012814 3782 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:44101
I20260809 07:54:35.013537 4503 ts_manager.cc:194] Registered new tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:35.015517 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:35.023998 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:35.024204 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3746
I20260809 07:54:35.025167 3827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:35.141587 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3746
I20260809 07:54:35.174533 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41169
--webserver_interface=127.26.37.62
--webserver_port=45307
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489 with env {}
W20260809 07:54:35.586910 4774 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:35.587307 4774 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:35.587457 4774 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:35.595831 4774 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:35.595960 4774 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:35.596093 4774 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:35.596180 4774 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:35.606647 4774 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=45307
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:35.608533 4774 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:35.610518 4774 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:35.619526 4783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:35.619457 4780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:35.619504 4779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:35.620585 4774 server_base.cc:1061] running on GCE node
I20260809 07:54:35.621399 4774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:35.622817 4774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:35.624059 4774 hybrid_clock.cc:648] HybridClock initialized: now 1786262075624024 us; error 63 us; skew 500 ppm
I20260809 07:54:35.624403 4774 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:35.626680 4774 webserver.cc:533] Webserver started at http://127.26.37.62:45307/ using document root <none> and password file <none>
I20260809 07:54:35.627445 4774 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:35.627612 4774 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:35.633230 4774 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:35.636240 4791 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:35.637676 4774 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:35.637854 4774 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "8cd6890316e440c7b2d9af6a446e0e1a"
format_stamp: "Formatted at 2026-08-09 07:54:31 on dist-test-slave-bw88"
I20260809 07:54:35.638615 4774 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:35.657784 4774 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:35.658675 4774 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:35.658973 4774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:35.679934 4774 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41169
I20260809 07:54:35.679958 4858 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41169 every 8 connection(s)
I20260809 07:54:35.681746 4774 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:35.688800 4859 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:35.690066 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4774
I20260809 07:54:35.690393 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4472
I20260809 07:54:35.692979 4859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Bootstrap starting.
I20260809 07:54:35.700460 4749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:35.713248 4859 log.cc:826] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:35.722963 4859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:35.723547 4859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: Bootstrap complete.
I20260809 07:54:35.730679 4859 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:35.731501 4859 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd6890316e440c7b2d9af6a446e0e1a, State: Initialized, Role: FOLLOWER
I20260809 07:54:35.732360 4859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:35.733287 4872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:35.733654 4872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:35.734443 4859 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:35.760383 4885 catalog_manager.cc:1295] Loaded cluster ID: 3eb3e5624e7c4e99a09250562e9872bd
I20260809 07:54:35.760612 4885 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: loading cluster ID for follower catalog manager: success
I20260809 07:54:35.765232 4885 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: acquiring CA information for follower catalog manager: success
I20260809 07:54:35.768291 4885 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:35.801186 4774 master_runner.cc:264] No leader master found from master 8cd6890316e440c7b2d9af6a446e0e1a
I20260809 07:54:35.801388 4774 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:54:35.823685 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4472
I20260809 07:54:35.850076 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34047
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--webserver_port=34429
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489 with env {}
W20260809 07:54:36.247381 4898 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:36.247721 4898 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:36.247804 4898 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:36.256412 4898 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:36.256582 4898 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:36.262738 4903 raft_consensus.cc:493] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8cd6890316e440c7b2d9af6a446e0e1a)
I20260809 07:54:36.262993 4903 raft_consensus.cc:515] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:36.264719 4903 leader_election.cc:290] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8cd6890316e440c7b2d9af6a446e0e1a (127.26.37.62:41169), dafd435bf9f441c2a7a25c35f1d23c1c (127.26.37.61:32783)
I20260809 07:54:36.265549 3894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "73489c7d312049f5adac35ca25ef23c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" is_pre_election: true
I20260809 07:54:36.268148 4898 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34047
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34429
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:36.269924 4906 raft_consensus.cc:493] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8cd6890316e440c7b2d9af6a446e0e1a)
I20260809 07:54:36.270076 4898 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:36.270157 4906 raft_consensus.cc:515] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:36.271937 4906 leader_election.cc:290] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8cd6890316e440c7b2d9af6a446e0e1a (127.26.37.62:41169), 73489c7d312049f5adac35ca25ef23c6 (127.26.37.60:36257)
I20260809 07:54:36.272722 4898 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:54:36.272703 3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "73489c7d312049f5adac35ca25ef23c6" is_pre_election: true
I20260809 07:54:36.272964 3997 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafd435bf9f441c2a7a25c35f1d23c1c in term 1.
I20260809 07:54:36.273173 4825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "73489c7d312049f5adac35ca25ef23c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" is_pre_election: true
I20260809 07:54:36.273726 4825 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73489c7d312049f5adac35ca25ef23c6 in term 1.
I20260809 07:54:36.273738 3864 leader_election.cc:304] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73489c7d312049f5adac35ca25ef23c6, dafd435bf9f441c2a7a25c35f1d23c1c; no voters:
I20260809 07:54:36.274094 4906 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:54:36.274204 4906 raft_consensus.cc:493] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8cd6890316e440c7b2d9af6a446e0e1a)
I20260809 07:54:36.274331 4906 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:54:36.274348 3962 leader_election.cc:304] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73489c7d312049f5adac35ca25ef23c6, 8cd6890316e440c7b2d9af6a446e0e1a; no voters: dafd435bf9f441c2a7a25c35f1d23c1c
I20260809 07:54:36.274995 4903 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:54:36.275125 4903 raft_consensus.cc:493] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8cd6890316e440c7b2d9af6a446e0e1a)
I20260809 07:54:36.275235 4903 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:54:36.277760 4906 raft_consensus.cc:515] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:36.278381 4903 raft_consensus.cc:515] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:36.279189 3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "73489c7d312049f5adac35ca25ef23c6"
I20260809 07:54:36.279451 3997 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dafd435bf9f441c2a7a25c35f1d23c1c in current term 2: Already voted for candidate 73489c7d312049f5adac35ca25ef23c6 in this term.
I20260809 07:54:36.280001 3894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "73489c7d312049f5adac35ca25ef23c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c"
I20260809 07:54:36.280081 4825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "73489c7d312049f5adac35ca25ef23c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "8cd6890316e440c7b2d9af6a446e0e1a"
I20260809 07:54:36.280300 3894 raft_consensus.cc:2393] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 73489c7d312049f5adac35ca25ef23c6 in current term 2: Already voted for candidate dafd435bf9f441c2a7a25c35f1d23c1c in this term.
I20260809 07:54:36.280306 4825 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:54:36.281481 4903 leader_election.cc:290] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [CANDIDATE]: Term 2 election: Requested vote from peers 8cd6890316e440c7b2d9af6a446e0e1a (127.26.37.62:41169), dafd435bf9f441c2a7a25c35f1d23c1c (127.26.37.61:32783)
I20260809 07:54:36.282226 4824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" is_pre_election: true
I20260809 07:54:36.282979 4823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "8cd6890316e440c7b2d9af6a446e0e1a"
I20260809 07:54:36.281292 4906 leader_election.cc:290] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [CANDIDATE]: Term 2 election: Requested vote from peers 8cd6890316e440c7b2d9af6a446e0e1a (127.26.37.62:41169), 73489c7d312049f5adac35ca25ef23c6 (127.26.37.60:36257)
I20260809 07:54:36.283948 3866 leader_election.cc:304] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dafd435bf9f441c2a7a25c35f1d23c1c; no voters: 73489c7d312049f5adac35ca25ef23c6, 8cd6890316e440c7b2d9af6a446e0e1a
I20260809 07:54:36.284412 4906 raft_consensus.cc:2749] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:54:36.287132 4825 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73489c7d312049f5adac35ca25ef23c6 in term 2.
W20260809 07:54:36.287657 4909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:36.287714 3962 leader_election.cc:304] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73489c7d312049f5adac35ca25ef23c6, 8cd6890316e440c7b2d9af6a446e0e1a; no voters: dafd435bf9f441c2a7a25c35f1d23c1c
W20260809 07:54:36.287734 4910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:36.288314 4903 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:54:36.288830 4903 raft_consensus.cc:697] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 2 LEADER]: Becoming Leader. State: Replica: 73489c7d312049f5adac35ca25ef23c6, State: Running, Role: LEADER
W20260809 07:54:36.289490 4913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:36.289413 4903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } }
I20260809 07:54:36.291167 4898 server_base.cc:1061] running on GCE node
I20260809 07:54:36.291887 4898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:36.291864 4915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73489c7d312049f5adac35ca25ef23c6. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:36.292184 4915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:36.292953 4918 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:54:36.293290 4898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:36.294584 4898 hybrid_clock.cc:648] HybridClock initialized: now 1786262076294534 us; error 30 us; skew 500 ppm
I20260809 07:54:36.295002 4898 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:36.295141 4918 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:54:36.296237 4918 catalog_manager.cc:1295] Loaded cluster ID: 3eb3e5624e7c4e99a09250562e9872bd
I20260809 07:54:36.296350 4918 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:54:36.297950 4898 webserver.cc:533] Webserver started at http://127.26.37.1:34429/ using document root <none> and password file <none>
I20260809 07:54:36.297991 4918 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:54:36.299015 4898 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:36.299072 4918 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: Loaded TSK: 0
I20260809 07:54:36.299175 4898 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:36.300133 4918 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:54:36.305553 4898 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:54:36.308298 4925 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:36.309850 4898 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:36.309993 4898 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2fad6d19572647f28e6c25d688eee5db"
format_stamp: "Formatted at 2026-08-09 07:54:33 on dist-test-slave-bw88"
I20260809 07:54:36.310711 4898 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:36.332361 4898 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:36.333206 4898 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:36.333478 4898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:36.334407 4898 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:36.336239 4898 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:36.336314 4898 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:36.336401 4898 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:36.336449 4898 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:36.373162 4898 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34047
I20260809 07:54:36.373224 5080 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34047 every 8 connection(s)
I20260809 07:54:36.374841 4898 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:36.383265 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 4898
I20260809 07:54:36.383584 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4898
I20260809 07:54:36.396783 5083 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41169
I20260809 07:54:36.397218 5083 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:36.398566 5083 heartbeater.cc:507] Master 127.26.37.62:41169 requested a full tablet report, sending...
I20260809 07:54:36.401782 4812 ts_manager.cc:194] Registered new tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:36.402096 5082 heartbeater.cc:344] Connected to a master server at 127.26.37.59:44489
I20260809 07:54:36.402276 5082 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:36.402958 5082 heartbeater.cc:507] Master 127.26.37.59:44489 requested a full tablet report, sending...
I20260809 07:54:36.404604 4503 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:36.411672 5081 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36257
I20260809 07:54:36.411854 5081 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:36.412479 5081 heartbeater.cc:507] Master 127.26.37.60:36257 requested a full tablet report, sending...
I20260809 07:54:36.413554 3981 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:36.415845 3981 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:46263
I20260809 07:54:36.417289 5084 heartbeater.cc:344] Connected to a master server at 127.26.37.61:32783
I20260809 07:54:36.417487 5084 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:36.417752 5075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:36.506534 5084 heartbeater.cc:507] Master 127.26.37.61:32783 requested a full tablet report, sending...
I20260809 07:54:36.507335 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4898
I20260809 07:54:36.510505 3882 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
W20260809 07:54:36.533403 3863 connection.cc:573] server connection from 127.26.37.1:51397 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:36.534278 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34047
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--webserver_port=34429
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489
--fault_before_start=1.0 with env {}
I20260809 07:54:36.698134 3894 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 2 FOLLOWER]: Refusing update from remote peer 73489c7d312049f5adac35ca25ef23c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 07:54:36.699018 4915 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:54:36.701164 4906 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73489c7d312049f5adac35ca25ef23c6. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:36.701527 4906 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:36.704200 4903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:36.704569 4903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:36.705013 4825 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 2 FOLLOWER]: Refusing update from remote peer 73489c7d312049f5adac35ca25ef23c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 07:54:36.705952 5107 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:54:36.713629 4906 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:36.713982 4906 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:36.714973 4514 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 1 LEARNER]: Advancing to term 2
I20260809 07:54:36.717790 5112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:36.718222 5112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:36.718632 5109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73489c7d312049f5adac35ca25ef23c6. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:36.718837 4514 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 2 LEARNER]: Refusing update from remote peer 73489c7d312049f5adac35ca25ef23c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 07:54:36.718952 5109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:36.719619 5107 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:54:36.723464 5116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:36.723932 5116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: This master's current role is: LEARNER
I20260809 07:54:36.724300 5107 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6: attempting to promote NON_VOTER 82c49856cb164f5fa90bdb87b4e08cdf to VOTER
I20260809 07:54:36.724252 5113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73489c7d312049f5adac35ca25ef23c6. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: true } } }
I20260809 07:54:36.724596 5113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: This master's current role is: LEARNER
I20260809 07:54:36.725718 5107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } }
I20260809 07:54:36.728106 3894 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 2 FOLLOWER]: Refusing update from remote peer 73489c7d312049f5adac35ca25ef23c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:54:36.728255 4514 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 2 LEARNER]: Refusing update from remote peer 73489c7d312049f5adac35ca25ef23c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:54:36.728897 4825 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 2 FOLLOWER]: Refusing update from remote peer 73489c7d312049f5adac35ca25ef23c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:54:36.728924 4903 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:54:36.729471 5107 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:54:36.729842 4903 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:54:36.732620 5107 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 82c49856cb164f5fa90bdb87b4e08cdf (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.733562 4825 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 82c49856cb164f5fa90bdb87b4e08cdf (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.733980 4514 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 82c49856cb164f5fa90bdb87b4e08cdf (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.738665 5109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.738682 5113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.738865 5109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd6890316e440c7b2d9af6a446e0e1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:36.738865 5113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c49856cb164f5fa90bdb87b4e08cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:36.741792 4915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.742039 4915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:36.742408 4915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.742631 4915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:36.742374 3894 raft_consensus.cc:2955] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 82c49856cb164f5fa90bdb87b4e08cdf (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.743427 5107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.743628 5107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:36.743528 4915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.743750 4915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73489c7d312049f5adac35ca25ef23c6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:36.745695 4906 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "73489c7d312049f5adac35ca25ef23c6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8cd6890316e440c7b2d9af6a446e0e1a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41169 } } peers { permanent_uuid: "dafd435bf9f441c2a7a25c35f1d23c1c" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 32783 } } peers { permanent_uuid: "73489c7d312049f5adac35ca25ef23c6" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 36257 } } peers { permanent_uuid: "82c49856cb164f5fa90bdb87b4e08cdf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 44489 } attrs { promote: false } } }
I20260809 07:54:36.745934 4906 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafd435bf9f441c2a7a25c35f1d23c1c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:54:36.917789 5103 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:36.918099 5103 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:36.918220 5103 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:54:36.918308 5103 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:36.926554 5103 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:36.926754 5103 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:36.937415 5103 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34047
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34429
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:36.939306 5103 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:36.941166 5103 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:36.951687 5122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:36.952225 5123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:36.952387 5126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:36.953089 5103 server_base.cc:1061] running on GCE node
I20260809 07:54:36.953840 5103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:36.954912 5103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:36.956144 5103 hybrid_clock.cc:648] HybridClock initialized: now 1786262076956086 us; error 29 us; skew 500 ppm
I20260809 07:54:36.956521 5103 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:36.959097 5103 webserver.cc:533] Webserver started at http://127.26.37.1:34429/ using document root <none> and password file <none>
I20260809 07:54:36.959919 5103 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:36.960096 5103 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:36.965862 5103 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:54:36.968730 5132 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:36.970249 5103 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:36.970430 5103 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2fad6d19572647f28e6c25d688eee5db"
format_stamp: "Formatted at 2026-08-09 07:54:33 on dist-test-slave-bw88"
I20260809 07:54:36.971197 5103 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:36.993541 5103 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:36.994469 5103 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:36.994781 5103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:36.995806 5103 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:36.997648 5103 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:36.997788 5103 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:36.997910 5103 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:36.998005 5103 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:54:36.998457 5103 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:54:36.998571 5103 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:54:36.998642 5103 fault_injection.cc:59] injecting fault for kudu (pid 5103): FLAGS_fault_before_start (process will exit)
I20260809 07:54:37.021720 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34047
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257
--webserver_port=34429
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489 with env {}
W20260809 07:54:37.381846 5141 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:37.382212 5141 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:37.382356 5141 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:37.390570 5141 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:37.390781 5141 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:37.400494 5141 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34047
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34429
--tserver_master_addrs=127.26.37.62:41169,127.26.37.61:32783,127.26.37.60:36257,127.26.37.59:44489
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:37.402279 5141 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:37.404109 5141 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:37.418147 5149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:37.418216 5150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:37.419281 5152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:37.420255 5141 server_base.cc:1061] running on GCE node
I20260809 07:54:37.420828 5141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:37.421859 5141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:37.423060 5141 hybrid_clock.cc:648] HybridClock initialized: now 1786262077423017 us; error 33 us; skew 500 ppm
I20260809 07:54:37.423383 5141 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:37.425681 5141 webserver.cc:533] Webserver started at http://127.26.37.1:34429/ using document root <none> and password file <none>
I20260809 07:54:37.426424 5141 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:37.426551 5141 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:37.432181 5141 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:54:37.435156 5160 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:37.436750 5141 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:54:37.436890 5141 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2fad6d19572647f28e6c25d688eee5db"
format_stamp: "Formatted at 2026-08-09 07:54:33 on dist-test-slave-bw88"
I20260809 07:54:37.437757 5141 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:37.470257 5141 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:37.471233 5141 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:37.471551 5141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:37.472571 5141 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:37.474442 5141 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:37.474535 5141 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:37.474710 5141 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:37.474761 5141 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:37.510166 5141 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34047
I20260809 07:54:37.510259 5315 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34047 every 8 connection(s)
I20260809 07:54:37.511819 5141 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:37.517848 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5141
I20260809 07:54:37.518199 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5141
I20260809 07:54:37.565083 5320 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41169
I20260809 07:54:37.565902 5320 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:37.567482 5320 heartbeater.cc:507] Master 127.26.37.62:41169 requested a full tablet report, sending...
I20260809 07:54:37.570403 4812 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:37.571599 5316 heartbeater.cc:344] Connected to a master server at 127.26.37.60:36257
I20260809 07:54:37.571750 5316 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:37.572235 5316 heartbeater.cc:507] Master 127.26.37.60:36257 requested a full tablet report, sending...
I20260809 07:54:37.572546 5321 heartbeater.cc:344] Connected to a master server at 127.26.37.61:32783
I20260809 07:54:37.572710 5321 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:37.572853 5319 heartbeater.cc:344] Connected to a master server at 127.26.37.59:44489
I20260809 07:54:37.573057 5319 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:37.573302 3981 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:37.573374 5321 heartbeater.cc:507] Master 127.26.37.61:32783 requested a full tablet report, sending...
I20260809 07:54:37.573642 5319 heartbeater.cc:507] Master 127.26.37.59:44489 requested a full tablet report, sending...
I20260809 07:54:37.574451 3981 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:43781
I20260809 07:54:37.574551 3882 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:37.575366 4503 ts_manager.cc:194] Re-registered known tserver with Master: 2fad6d19572647f28e6c25d688eee5db (127.26.37.1:34047)
I20260809 07:54:37.578509 5310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:37.666874 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5141
I20260809 07:54:37.693868 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4774
I20260809 07:54:37.700524 4851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:37.810979 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4774
W20260809 07:54:37.838449 3866 connection.cc:573] client connection to 127.26.37.62:41169 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:37.838792 3844 master_runner.cc:217] Error getting master consensus for 127.26.37.62:41169: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:37.839084 3844 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:54:37.839227 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3844
I20260809 07:54:37.840513 3927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:37.951731 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3844
I20260809 07:54:37.981551 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 3937
I20260809 07:54:37.982834 4026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:38.095952 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 3937
I20260809 07:54:38.127360 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 4308
I20260809 07:54:38.128520 4537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:38.241328 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 4308
2026-08-09T07:54:38Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/1 (7290 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/2
2026-08-09T07:54:38Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:38Z Disabled control of system clock
2026-08-09T07:54:38Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:38Z Disabled control of system clock
2026-08-09T07:54:38Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:38Z Disabled control of system clock
2026-08-09T07:54:38Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:38Z Disabled control of system clock
2026-08-09T07:54:38Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:54:38Z Disabled control of system clock
I20260809 07:54:38.530022 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39799
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289 with env {}
W20260809 07:54:38.893263 5354 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:38.893725 5354 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:38.893893 5354 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:38.902500 5354 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:38.902619 5354 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:38.902756 5354 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:38.902844 5354 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:38.912981 5354 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:38.914713 5354 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:38.916610 5354 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:38.924834 5362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:38.924868 5363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:38.925556 5365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:38.926254 5354 server_base.cc:1061] running on GCE node
I20260809 07:54:38.926988 5354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:38.928541 5354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:38.929781 5354 hybrid_clock.cc:648] HybridClock initialized: now 1786262078929654 us; error 97 us; skew 500 ppm
I20260809 07:54:38.930164 5354 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:38.932269 5354 webserver.cc:533] Webserver started at http://127.26.37.62:45967/ using document root <none> and password file <none>
I20260809 07:54:38.933020 5354 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:38.933198 5354 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:38.933655 5354 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:38.935983 5354 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "f24efeb90e8c4810b0e68d760a623266"
format_stamp: "Formatted at 2026-08-09 07:54:38 on dist-test-slave-bw88"
I20260809 07:54:38.937096 5354 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "f24efeb90e8c4810b0e68d760a623266"
format_stamp: "Formatted at 2026-08-09 07:54:38 on dist-test-slave-bw88"
I20260809 07:54:38.942658 5354 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 07:54:38.946128 5373 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:38.947474 5354 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:54:38.947656 5354 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "f24efeb90e8c4810b0e68d760a623266"
format_stamp: "Formatted at 2026-08-09 07:54:38 on dist-test-slave-bw88"
I20260809 07:54:38.947883 5354 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:38.977972 5354 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:38.978945 5354 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:38.979293 5354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:39.000669 5354 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39799
I20260809 07:54:39.000711 5437 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39799 every 8 connection(s)
I20260809 07:54:39.002532 5354 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:39.007182 5438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:39.010500 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5354
I20260809 07:54:39.010711 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:54:39.012158 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39407
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289 with env {}
I20260809 07:54:39.014932 5438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:39.031850 5438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:39.033197 5375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39407: connect: Connection refused (error 111)
W20260809 07:54:39.034823 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39407: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39407: connect: Connection refused (error 111)
I20260809 07:54:39.084167 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } attempt: 1
W20260809 07:54:39.086689 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39407: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39407: connect: Connection refused (error 111)
I20260809 07:54:39.154956 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } attempt: 2
W20260809 07:54:39.157058 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39407: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39407: connect: Connection refused (error 111)
I20260809 07:54:39.248446 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } attempt: 3
W20260809 07:54:39.250651 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39407: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39407: connect: Connection refused (error 111)
W20260809 07:54:39.378556 5442 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:39.378918 5442 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:39.378994 5442 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:39.386972 5442 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:39.387060 5442 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:39.387116 5442 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:39.387161 5442 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:39.393945 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } attempt: 4
W20260809 07:54:39.396099 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:39407: Network error: Client connection negotiation failed: client connection to 127.26.37.61:39407: connect: Connection refused (error 111)
I20260809 07:54:39.397164 5442 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:39407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:39.398842 5442 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:39.400828 5442 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:39.408984 5451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:39.409063 5454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:39.409232 5452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:39.409993 5442 server_base.cc:1061] running on GCE node
I20260809 07:54:39.410725 5442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:39.412412 5442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:39.413645 5442 hybrid_clock.cc:648] HybridClock initialized: now 1786262079413482 us; error 132 us; skew 500 ppm
I20260809 07:54:39.413995 5442 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:39.416095 5442 webserver.cc:533] Webserver started at http://127.26.37.61:44355/ using document root <none> and password file <none>
I20260809 07:54:39.416836 5442 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:39.416975 5442 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:39.417414 5442 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:39.419811 5442 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "a53948f086e84784a6d898a4e5eea0a9"
format_stamp: "Formatted at 2026-08-09 07:54:39 on dist-test-slave-bw88"
I20260809 07:54:39.420668 5442 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "a53948f086e84784a6d898a4e5eea0a9"
format_stamp: "Formatted at 2026-08-09 07:54:39 on dist-test-slave-bw88"
I20260809 07:54:39.426935 5442 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260809 07:54:39.430917 5462 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:39.432397 5442 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:54:39.432621 5442 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "a53948f086e84784a6d898a4e5eea0a9"
format_stamp: "Formatted at 2026-08-09 07:54:39 on dist-test-slave-bw88"
I20260809 07:54:39.432844 5442 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:39.456146 5442 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:39.457036 5442 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:39.457458 5442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:39.478964 5442 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:39407
I20260809 07:54:39.478984 5528 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:39407 every 8 connection(s)
I20260809 07:54:39.480589 5442 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:54:39.482894 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5442
I20260809 07:54:39.483119 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:54:39.484145 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:46289
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289 with env {}
I20260809 07:54:39.485280 5530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:39.490914 5530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:39.505609 5530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:39.515201 5530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:39.516405 5463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
W20260809 07:54:39.518258 5530 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:54:39.567611 5530 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 1
W20260809 07:54:39.569885 5530 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:54:39.638118 5530 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 2
W20260809 07:54:39.640312 5530 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:54:39.695393 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } attempt: 5
I20260809 07:54:39.702613 5438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:54:39.704636 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:54:39.731554 5530 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 3
W20260809 07:54:39.733898 5530 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:54:39.755915 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 1
W20260809 07:54:39.758316 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:54:39.826611 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 2
W20260809 07:54:39.828707 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
W20260809 07:54:39.848232 5536 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:39.848557 5536 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:39.848627 5536 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:39.856810 5536 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:39.856899 5536 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:39.856973 5536 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:39.857014 5536 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:39.866894 5536 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:46289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:39.868714 5536 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:39.870760 5536 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:54:39.877130 5530 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 4
W20260809 07:54:39.878868 5549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:39.879644 5548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:39.880116 5530 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
W20260809 07:54:39.881392 5551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:39.881829 5536 server_base.cc:1061] running on GCE node
I20260809 07:54:39.882570 5536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:39.884052 5536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:39.885257 5536 hybrid_clock.cc:648] HybridClock initialized: now 1786262079885137 us; error 88 us; skew 500 ppm
I20260809 07:54:39.885661 5536 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:39.887815 5536 webserver.cc:533] Webserver started at http://127.26.37.60:33585/ using document root <none> and password file <none>
I20260809 07:54:39.888535 5536 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:39.888655 5536 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:39.889027 5536 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:39.891426 5536 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "1e8f893dde3c49099e2f7d5c7a321b3c"
format_stamp: "Formatted at 2026-08-09 07:54:39 on dist-test-slave-bw88"
I20260809 07:54:39.892246 5536 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "1e8f893dde3c49099e2f7d5c7a321b3c"
format_stamp: "Formatted at 2026-08-09 07:54:39 on dist-test-slave-bw88"
I20260809 07:54:39.897684 5536 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:54:39.900946 5561 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:39.902350 5536 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:39.902551 5536 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "1e8f893dde3c49099e2f7d5c7a321b3c"
format_stamp: "Formatted at 2026-08-09 07:54:39 on dist-test-slave-bw88"
I20260809 07:54:39.902776 5536 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:39.920190 5536 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:39.921104 5536 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:39.921442 5536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:39.934988 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 3
W20260809 07:54:39.937222 5438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:54:39.942689 5536 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:46289
I20260809 07:54:39.942699 5629 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:46289 every 8 connection(s)
I20260809 07:54:39.944291 5536 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:54:39.948510 5630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:39.953222 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5536
I20260809 07:54:39.953467 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:54:39.954493 5630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:39.969666 5630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:39.976347 5630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:39.986987 5630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: Bootstrap starting.
I20260809 07:54:39.990077 5630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:39.991106 5630 log.cc:826] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:39.994099 5630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: No bootstrap required, opened a new log
I20260809 07:54:39.999305 5630 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:39.999562 5630 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:39.999662 5630 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e8f893dde3c49099e2f7d5c7a321b3c, State: Initialized, Role: FOLLOWER
I20260809 07:54:40.000375 5630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:40.001308 5639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.001672 5639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:40.002671 5630 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:40.010540 5536 master_runner.cc:187] Error getting master registration for 127.26.37.62:39799: OK, instance_id { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" instance_seqno: 1786262078995071 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:54:40.016373 5536 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:54:40.018992 5652 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:40.019116 5652 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:40.114501 5438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 4
I20260809 07:54:40.126989 5438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: Bootstrap starting.
I20260809 07:54:40.130441 5438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:40.131615 5438 log.cc:826] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:40.134531 5438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: No bootstrap required, opened a new log
I20260809 07:54:40.141012 5438 raft_consensus.cc:359] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:40.141412 5438 raft_consensus.cc:385] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:40.141573 5438 raft_consensus.cc:740] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24efeb90e8c4810b0e68d760a623266, State: Initialized, Role: FOLLOWER
I20260809 07:54:40.142190 5438 consensus_queue.cc:260] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:40.143262 5656 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.143573 5656 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:40.144404 5438 sys_catalog.cc:565] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:54:40.155411 5668 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:40.155566 5668 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:40.154940 5354 master_runner.cc:187] Error getting master registration for 127.26.37.61:39407: OK, instance_id { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" instance_seqno: 1786262079473340 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:54:40.164208 5354 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:54:40.173633 5569 tablet.cc:2406] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:54:40.179478 5530 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 5
I20260809 07:54:40.181361 5639 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:54:40.181674 5639 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:40.183672 5639 leader_election.cc:290] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f24efeb90e8c4810b0e68d760a623266 (127.26.37.62:39799), a53948f086e84784a6d898a4e5eea0a9 (127.26.37.61:39407)
I20260809 07:54:40.185719 5409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24efeb90e8c4810b0e68d760a623266" is_pre_election: true
I20260809 07:54:40.186338 5409 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e8f893dde3c49099e2f7d5c7a321b3c in term 0.
I20260809 07:54:40.187623 5566 leader_election.cc:304] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e8f893dde3c49099e2f7d5c7a321b3c, f24efeb90e8c4810b0e68d760a623266; no voters:
I20260809 07:54:40.188164 5639 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:54:40.188326 5639 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:54:40.188457 5639 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:40.192045 5639 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:40.193322 5409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24efeb90e8c4810b0e68d760a623266"
I20260809 07:54:40.193563 5409 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:40.195272 5530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9: Bootstrap starting.
I20260809 07:54:40.196931 5409 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e8f893dde3c49099e2f7d5c7a321b3c in term 1.
I20260809 07:54:40.197523 5566 leader_election.cc:304] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e8f893dde3c49099e2f7d5c7a321b3c, f24efeb90e8c4810b0e68d760a623266; no voters:
I20260809 07:54:40.199385 5530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:40.200764 5530 log.cc:826] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:40.201648 5639 leader_election.cc:290] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [CANDIDATE]: Term 1 election: Requested vote from peers f24efeb90e8c4810b0e68d760a623266 (127.26.37.62:39799), a53948f086e84784a6d898a4e5eea0a9 (127.26.37.61:39407)
I20260809 07:54:40.201716 5670 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:54:40.202415 5670 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 1 LEADER]: Becoming Leader. State: Replica: 1e8f893dde3c49099e2f7d5c7a321b3c, State: Running, Role: LEADER
I20260809 07:54:40.203634 5670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:40.209617 5530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9: No bootstrap required, opened a new log
I20260809 07:54:40.222096 5639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e8f893dde3c49099e2f7d5c7a321b3c. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.222344 5639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:40.224056 5530 raft_consensus.cc:359] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:40.225178 5530 raft_consensus.cc:385] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:40.225735 5530 raft_consensus.cc:740] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53948f086e84784a6d898a4e5eea0a9, State: Initialized, Role: FOLLOWER
I20260809 07:54:40.228297 5530 consensus_queue.cc:260] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:54:40.230055 5673 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
W20260809 07:54:40.232335 5383 tablet.cc:2406] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:54:40.232926 5673 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:54:40.239401 5497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53948f086e84784a6d898a4e5eea0a9" is_pre_election: true
I20260809 07:54:40.239849 5496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53948f086e84784a6d898a4e5eea0a9"
I20260809 07:54:40.240198 5496 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:54:40.243819 5496 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e8f893dde3c49099e2f7d5c7a321b3c in term 1.
I20260809 07:54:40.243886 5409 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 1 FOLLOWER]: Refusing update from remote peer 1e8f893dde3c49099e2f7d5c7a321b3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:54:40.244509 5495 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 1 FOLLOWER]: Refusing update from remote peer 1e8f893dde3c49099e2f7d5c7a321b3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:54:40.244992 5639 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:54:40.247735 5497 raft_consensus.cc:2376] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1e8f893dde3c49099e2f7d5c7a321b3c in term 1. Re-sending same reply.
I20260809 07:54:40.248387 5639 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:54:40.249512 5530 sys_catalog.cc:565] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:40.268996 5656 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e8f893dde3c49099e2f7d5c7a321b3c. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.269273 5656 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:40.279315 5679 mvcc.cc:204] Tried to move back new op lower bound from 7316529480665190400 to 7316529480528056320. Current Snapshot: MvccSnapshot[applied={T|T < 7316529480665190400}]
I20260809 07:54:40.286275 5676 sys_catalog.cc:455] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.286715 5676 sys_catalog.cc:458] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:40.287125 5680 sys_catalog.cc:455] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e8f893dde3c49099e2f7d5c7a321b3c. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.287441 5680 sys_catalog.cc:458] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:40.288995 5670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.289297 5670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:40.289944 5656 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.290156 5656 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:40.294701 5639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.294971 5639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:40.295598 5680 sys_catalog.cc:455] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:54:40.295853 5680 sys_catalog.cc:458] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:40.305390 5673 catalog_manager.cc:1383] Generated new cluster ID: 1cc4c017759e4c39a1884a11ea9fe0dc
I20260809 07:54:40.305516 5673 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:54:40.307312 5685 mvcc.cc:204] Tried to move back new op lower bound from 7316529480665190400 to 7316529480528056320. Current Snapshot: MvccSnapshot[applied={T|T < 7316529480665190400}]
I20260809 07:54:40.355007 5673 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:54:40.356853 5673 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:54:40.367097 5706 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:54:40.382848 5673 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: Generated new TSK 0
I20260809 07:54:40.383955 5673 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:54:40.398401 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:54:40.798764 5707 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:40.799074 5707 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:40.799170 5707 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:40.808218 5707 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:40.808377 5707 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:40.818564 5707 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:40.820225 5707 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:40.822141 5707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:40.832015 5719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:40.832015 5718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:40.832827 5724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:40.833244 5707 server_base.cc:1061] running on GCE node
I20260809 07:54:40.833827 5707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:40.834980 5707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:40.836187 5707 hybrid_clock.cc:648] HybridClock initialized: now 1786262080836076 us; error 78 us; skew 500 ppm
I20260809 07:54:40.836520 5707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:40.838843 5707 webserver.cc:533] Webserver started at http://127.26.37.1:36423/ using document root <none> and password file <none>
I20260809 07:54:40.839614 5707 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:40.839741 5707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:40.840132 5707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:40.842746 5707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "09cd829aea0b4ff0b349c901223449bc"
format_stamp: "Formatted at 2026-08-09 07:54:40 on dist-test-slave-bw88"
I20260809 07:54:40.843578 5707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "09cd829aea0b4ff0b349c901223449bc"
format_stamp: "Formatted at 2026-08-09 07:54:40 on dist-test-slave-bw88"
I20260809 07:54:40.849293 5707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 07:54:40.853070 5735 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:40.854632 5707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:40.854817 5707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "09cd829aea0b4ff0b349c901223449bc"
format_stamp: "Formatted at 2026-08-09 07:54:40 on dist-test-slave-bw88"
I20260809 07:54:40.855060 5707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:40.902714 5707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:40.903697 5707 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:40.904027 5707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:40.905113 5707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:40.906901 5707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:40.906992 5707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:40.907195 5707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:40.907281 5707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:40.945106 5707 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:42449
I20260809 07:54:40.945294 5890 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:42449 every 8 connection(s)
I20260809 07:54:40.946949 5707 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:40.954329 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5707
I20260809 07:54:40.954581 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:54:40.978130 5891 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46289
I20260809 07:54:40.978566 5891 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:40.979801 5891 heartbeater.cc:507] Master 127.26.37.60:46289 requested a full tablet report, sending...
I20260809 07:54:40.979962 5895 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39407
I20260809 07:54:40.980106 5895 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:40.980577 5895 heartbeater.cc:507] Master 127.26.37.61:39407 requested a full tablet report, sending...
I20260809 07:54:40.980861 5894 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39799
I20260809 07:54:40.981014 5894 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:40.981630 5894 heartbeater.cc:507] Master 127.26.37.62:39799 requested a full tablet report, sending...
I20260809 07:54:40.982585 5487 ts_manager.cc:194] Registered new tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:40.982592 5582 ts_manager.cc:194] Registered new tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:40.983332 5398 ts_manager.cc:194] Registered new tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:40.984499 5582 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:42137
I20260809 07:54:40.996965 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:40.999395 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:39783
--webserver_interface=127.26.37.59
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783 with env {}
I20260809 07:54:41.158088 5668 catalog_manager.cc:1295] Loaded cluster ID: 1cc4c017759e4c39a1884a11ea9fe0dc
I20260809 07:54:41.158210 5668 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: loading cluster ID for follower catalog manager: success
I20260809 07:54:41.162221 5668 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: acquiring CA information for follower catalog manager: success
I20260809 07:54:41.164702 5668 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:54:41.365592 5904 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:41.365908 5904 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:41.365983 5904 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:54:41.373708 5706 catalog_manager.cc:1295] Loaded cluster ID: 1cc4c017759e4c39a1884a11ea9fe0dc
I20260809 07:54:41.373850 5706 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9: loading cluster ID for follower catalog manager: success
I20260809 07:54:41.377841 5706 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9: acquiring CA information for follower catalog manager: success
W20260809 07:54:41.378417 5904 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:41.378532 5904 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:41.378644 5904 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:41.378723 5904 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:41.380474 5706 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:41.389642 5904 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:39783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:41.391260 5904 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:41.393124 5904 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:41.401326 5914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:41.401439 5913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:41.402374 5916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:41.403935 5904 server_base.cc:1061] running on GCE node
I20260809 07:54:41.404731 5904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:41.406385 5904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:41.407792 5904 hybrid_clock.cc:648] HybridClock initialized: now 1786262081407646 us; error 114 us; skew 500 ppm
I20260809 07:54:41.408151 5904 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:41.410434 5904 webserver.cc:533] Webserver started at http://127.26.37.59:40423/ using document root <none> and password file <none>
I20260809 07:54:41.411154 5904 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:41.411289 5904 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:41.411705 5904 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:54:41.414167 5904 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "8f640506aeb24d8ba3741e393485184e"
format_stamp: "Formatted at 2026-08-09 07:54:41 on dist-test-slave-bw88"
I20260809 07:54:41.414995 5904 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "8f640506aeb24d8ba3741e393485184e"
format_stamp: "Formatted at 2026-08-09 07:54:41 on dist-test-slave-bw88"
I20260809 07:54:41.420871 5904 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:54:41.424563 5925 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:41.426221 5904 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:54:41.426438 5904 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "8f640506aeb24d8ba3741e393485184e"
format_stamp: "Formatted at 2026-08-09 07:54:41 on dist-test-slave-bw88"
I20260809 07:54:41.426689 5904 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:41.452144 5904 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:41.453208 5904 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:41.453538 5904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:41.475875 5904 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:39783
I20260809 07:54:41.475919 5991 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:39783 every 8 connection(s)
I20260809 07:54:41.478387 5904 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:54:41.486299 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 5904
I20260809 07:54:41.496584 5992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:54:41.506665 5992 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:41.519830 5992 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:41.526639 5992 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:41.533097 5992 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:54:41.545102 5992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Bootstrap starting.
I20260809 07:54:41.549523 5992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Neither blocks nor log segments found. Creating new log.
I20260809 07:54:41.550984 5992 log.cc:826] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:41.554327 5992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: No bootstrap required, opened a new log
I20260809 07:54:41.562145 5992 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } }
I20260809 07:54:41.562566 5992 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:54:41.562698 5992 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f640506aeb24d8ba3741e393485184e, State: Initialized, Role: FOLLOWER
I20260809 07:54:41.563498 5992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } }
I20260809 07:54:41.564611 6004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } } }
I20260809 07:54:41.565085 6004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:41.565924 5992 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:54:41.576186 6018 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:54:41.576329 6018 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:54:41.584245 5904 master_runner.cc:429] Detected that this master 8f640506aeb24d8ba3741e393485184e is joining an existing cluster
I20260809 07:54:41.584357 5904 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:54:41.585466 5904 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:54:41.585817 5904 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:54:41.586071 5904 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: stopping tablet replica
I20260809 07:54:41.588166 5904 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:39783
I20260809 07:54:41.640728 5582 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:39783: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8f640506aeb24d8ba3741e393485184e"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 39783
}
attrs {
promote: true
}
}
dest_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c"
cas_config_opid_index: -1
I20260809 07:54:41.642506 5582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } }
W20260809 07:54:41.646154 5564 consensus_peers.cc:597] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c -> Peer 8f640506aeb24d8ba3741e393485184e (127.26.37.59:39783): Couldn't send request to peer 8f640506aeb24d8ba3741e393485184e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:54:41.647375 5409 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 1 FOLLOWER]: Refusing update from remote peer 1e8f893dde3c49099e2f7d5c7a321b3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:54:41.647542 5497 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 1 FOLLOWER]: Refusing update from remote peer 1e8f893dde3c49099e2f7d5c7a321b3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:54:41.648236 5996 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:54:41.648754 5996 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:54:41.662016 5996 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8f640506aeb24d8ba3741e393485184e (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:41.666311 5996 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:39783
I20260809 07:54:41.667111 5904 master.cc:562] Master@127.26.37.59:39783 shutting down...
I20260809 07:54:41.669937 5409 raft_consensus.cc:2955] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8f640506aeb24d8ba3741e393485184e (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:41.682440 6037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:41.682840 6037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:41.698432 6036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:41.698861 6036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:41.701459 6032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:41.702013 6032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
W20260809 07:54:41.718653 5342 connection.cc:573] client connection to 127.26.37.59:39783 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:41.714958 5497 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8f640506aeb24d8ba3741e393485184e (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:41.721660 5904 master.cc:584] Master@127.26.37.59:39783 shutdown complete.
I20260809 07:54:41.721822 5904 master_runner.cc:306] Clearing existing system tablet
I20260809 07:54:41.722316 5904 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:41.722479 5904 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:41.732517 6041 sys_catalog.cc:455] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:41.732904 6041 sys_catalog.cc:458] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:41.733742 5904 fs_manager.cc:714] Time spent opening directory manager: real 0.010s user 0.010s sys 0.001s
I20260809 07:54:41.735891 6043 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:41.736307 5904 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:41.736588 5904 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 07:54:41.736701 5904 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "8f640506aeb24d8ba3741e393485184e"
format_stamp: "Formatted at 2026-08-09 07:54:41 on dist-test-slave-bw88"
I20260809 07:54:41.741379 5904 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:54:41.748626 5904 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:54:41.748793 5904 log.cc:1199] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:54:41.749294 5904 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Deleting consensus metadata
I20260809 07:54:41.749575 5904 master_runner.cc:316] Copying system tablet from 127.26.37.60:46289
I20260809 07:54:41.752578 5904 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.60:46289
I20260809 07:54:41.764889 5608 tablet_copy_service.cc:140] P 1e8f893dde3c49099e2f7d5c7a321b3c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8f640506aeb24d8ba3741e393485184e ({username='slave'} at 127.0.0.1:35298)
I20260809 07:54:41.765213 5608 tablet_copy_service.cc:161] P 1e8f893dde3c49099e2f7d5c7a321b3c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8f640506aeb24d8ba3741e393485184e at {username='slave'} at 127.0.0.1:35298: session id = 8f640506aeb24d8ba3741e393485184e-00000000000000000000000000000000
I20260809 07:54:41.769963 5608 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:54:41.772578 5904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:54:41.780118 5904 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: tablet copy: Starting download of 0 data blocks...
I20260809 07:54:41.780465 5904 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: tablet copy: Starting download of 1 WAL segments...
I20260809 07:54:41.783527 5904 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:54:41.788261 5608 tablet_copy_service.cc:342] P 1e8f893dde3c49099e2f7d5c7a321b3c: Request end of tablet copy session 8f640506aeb24d8ba3741e393485184e-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:35298
I20260809 07:54:41.788497 5608 tablet_copy_service.cc:434] P 1e8f893dde3c49099e2f7d5c7a321b3c: ending tablet copy session 8f640506aeb24d8ba3741e393485184e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8f640506aeb24d8ba3741e393485184e
W20260809 07:54:41.797670 5904 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260809 07:54:41.802399 5904 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:41.805305 6053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:41.805692 6055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:41.812681 6059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:41.813982 5904 server_base.cc:1061] running on GCE node
I20260809 07:54:41.814306 5904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:41.815178 5904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:41.816393 5904 hybrid_clock.cc:648] HybridClock initialized: now 1786262081816224 us; error 130 us; skew 500 ppm
I20260809 07:54:41.816637 5904 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:41.818146 5904 webserver.cc:533] Webserver started at http://127.26.37.59:36723/ using document root <none> and password file <none>
I20260809 07:54:41.818547 5904 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:41.818655 5904 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:41.826226 5904 fs_manager.cc:714] Time spent opening directory manager: real 0.007s user 0.002s sys 0.006s
I20260809 07:54:41.828454 6067 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:41.828959 5904 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 07:54:41.829135 5904 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "8f640506aeb24d8ba3741e393485184e"
format_stamp: "Formatted at 2026-08-09 07:54:41 on dist-test-slave-bw88"
I20260809 07:54:41.829375 5904 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:41.858096 5904 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:41.859037 5904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:41.889946 5904 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:39783
I20260809 07:54:41.889971 6130 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:39783 every 8 connection(s)
I20260809 07:54:41.891187 5904 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:54:41.894762 6131 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:41.897063 6131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Bootstrap starting.
I20260809 07:54:41.910490 6131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:41.910949 6131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: Bootstrap complete.
I20260809 07:54:41.911850 6131 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } }
I20260809 07:54:41.912083 6131 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8f640506aeb24d8ba3741e393485184e, State: Initialized, Role: LEARNER
I20260809 07:54:41.912330 6131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } }
I20260809 07:54:41.913266 6131 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:41.913110 6135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:41.913467 6135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: This master's current role is: LEARNER
I20260809 07:54:41.921756 6148 catalog_manager.cc:1295] Loaded cluster ID: 1cc4c017759e4c39a1884a11ea9fe0dc
I20260809 07:54:41.921857 6148 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: loading cluster ID for follower catalog manager: success
I20260809 07:54:41.926255 6148 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: acquiring CA information for follower catalog manager: success
I20260809 07:54:41.928828 6148 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:41.932614 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5707
I20260809 07:54:41.940677 5886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:42.071632 5891 heartbeater.cc:499] Master 127.26.37.60:46289 was elected leader, sending a full tablet report...
I20260809 07:54:42.074096 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5707
I20260809 07:54:42.095959 6098 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:54:42.097245 6135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e8f893dde3c49099e2f7d5c7a321b3c. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:42.097553 6135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: This master's current role is: LEARNER
W20260809 07:54:42.101759 5564 connection.cc:573] server connection from 127.26.37.1:42137 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:54:42.101861 5463 connection.cc:573] server connection from 127.26.37.1:60167 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:42.103338 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:42449
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--webserver_port=36423
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783 with env {}
I20260809 07:54:42.164469 6036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:42.164825 6036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
W20260809 07:54:42.470988 6153 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:42.471329 6153 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:42.471506 6153 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:42.480975 6153 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:42.481248 6153 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:42.497507 6153 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:42449
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=36423
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:42.499161 6153 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:42.500973 6153 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:54:42.505821 6038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: true } } }
I20260809 07:54:42.506078 6038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:42.507676 6036 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c: attempting to promote NON_VOTER 8f640506aeb24d8ba3741e393485184e to VOTER
I20260809 07:54:42.508649 6036 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } }
W20260809 07:54:42.511627 6161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:42.511940 5497 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 1 FOLLOWER]: Refusing update from remote peer 1e8f893dde3c49099e2f7d5c7a321b3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:54:42.511940 5409 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 1 FOLLOWER]: Refusing update from remote peer 1e8f893dde3c49099e2f7d5c7a321b3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:54:42.512642 5996 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:54:42.512939 6098 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 1 LEARNER]: Refusing update from remote peer 1e8f893dde3c49099e2f7d5c7a321b3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:54:42.513270 5996 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:54:42.513803 5996 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 07:54:42.514307 6163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:42.517094 6165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:42.517390 6153 server_base.cc:1061] running on GCE node
I20260809 07:54:42.517324 5996 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8f640506aeb24d8ba3741e393485184e (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.518175 6153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:42.518190 5409 raft_consensus.cc:2955] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8f640506aeb24d8ba3741e393485184e (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.522696 6038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.523039 6038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:42.523326 6153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:42.523326 6038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.523514 6038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:42.523049 5996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.523938 5996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:42.524243 5497 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8f640506aeb24d8ba3741e393485184e (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.525825 6173 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.526093 6173 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:42.526713 6153 hybrid_clock.cc:648] HybridClock initialized: now 1786262082526385 us; error 287 us; skew 500 ppm
I20260809 07:54:42.527104 6153 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:42.527936 6036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.528128 6036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:42.528925 6174 sys_catalog.cc:455] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.529208 6174 sys_catalog.cc:458] T 00000000000000000000000000000000 P a53948f086e84784a6d898a4e5eea0a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:42.529218 6098 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8f640506aeb24d8ba3741e393485184e (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.530798 6153 webserver.cc:533] Webserver started at http://127.26.37.1:36423/ using document root <none> and password file <none>
I20260809 07:54:42.531790 6153 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:42.531940 6153 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:42.534061 6135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:42.534313 6135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f640506aeb24d8ba3741e393485184e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:42.538332 6153 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:54:42.540942 6179 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:42.542410 6153 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:42.542603 6153 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "09cd829aea0b4ff0b349c901223449bc"
format_stamp: "Formatted at 2026-08-09 07:54:40 on dist-test-slave-bw88"
I20260809 07:54:42.543358 6153 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:42.560953 6153 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:42.561935 6153 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:42.562244 6153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:42.563261 6153 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:42.565055 6153 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:42.565162 6153 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:42.565318 6153 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:42.565419 6153 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 07:54:42.599417 6153 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:42449
I20260809 07:54:42.599454 6323 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:42449 every 8 connection(s)
I20260809 07:54:42.600967 6153 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:42.604050 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6153
I20260809 07:54:42.636528 6324 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46289
I20260809 07:54:42.636587 6327 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39799
I20260809 07:54:42.636924 6324 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:42.636958 6328 heartbeater.cc:344] Connected to a master server at 127.26.37.59:39783
I20260809 07:54:42.636931 6327 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:42.637153 6328 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:42.637703 6329 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39407
I20260809 07:54:42.638036 6329 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:42.638486 6324 heartbeater.cc:507] Master 127.26.37.60:46289 requested a full tablet report, sending...
I20260809 07:54:42.638522 6327 heartbeater.cc:507] Master 127.26.37.62:39799 requested a full tablet report, sending...
I20260809 07:54:42.638574 6328 heartbeater.cc:507] Master 127.26.37.59:39783 requested a full tablet report, sending...
I20260809 07:54:42.638628 6329 heartbeater.cc:507] Master 127.26.37.61:39407 requested a full tablet report, sending...
I20260809 07:54:42.640587 5487 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:42.640606 5582 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:42.640611 5398 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:42.641014 6087 ts_manager.cc:194] Registered new tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:42.641619 5582 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:41907
I20260809 07:54:42.644626 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:42.652148 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:54:42.652335 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5354
I20260809 07:54:42.653230 5432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:42.760504 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5354
I20260809 07:54:42.790951 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39799
--webserver_interface=127.26.37.62
--webserver_port=45967
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783 with env {}
W20260809 07:54:42.966771 5566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:39799: connect: Connection refused (error 111)
W20260809 07:54:42.970688 5566 consensus_peers.cc:597] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c -> Peer f24efeb90e8c4810b0e68d760a623266 (127.26.37.62:39799): Couldn't send request to peer f24efeb90e8c4810b0e68d760a623266. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:39799: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:54:43.171393 6343 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:43.171783 6343 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:43.171913 6343 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:43.180294 6343 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:54:43.180446 6343 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:43.180568 6343 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:54:43.180649 6343 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:54:43.190855 6343 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=45967
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:43.192649 6343 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:43.194579 6343 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:43.202852 6354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:43.203009 6351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:43.204463 6356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:43.204785 6343 server_base.cc:1061] running on GCE node
I20260809 07:54:43.205606 6343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:43.207203 6343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:43.208491 6343 hybrid_clock.cc:648] HybridClock initialized: now 1786262083208292 us; error 167 us; skew 500 ppm
I20260809 07:54:43.208834 6343 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:43.210960 6343 webserver.cc:533] Webserver started at http://127.26.37.62:45967/ using document root <none> and password file <none>
I20260809 07:54:43.211669 6343 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:43.211843 6343 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:43.217242 6343 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:54:43.220042 6363 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:43.221426 6343 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:43.221612 6343 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "f24efeb90e8c4810b0e68d760a623266"
format_stamp: "Formatted at 2026-08-09 07:54:38 on dist-test-slave-bw88"
I20260809 07:54:43.222271 6343 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:43.240473 6343 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:43.241291 6343 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:43.241619 6343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:43.262696 6343 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39799
I20260809 07:54:43.262718 6428 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39799 every 8 connection(s)
I20260809 07:54:43.264513 6343 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:54:43.265815 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6343
I20260809 07:54:43.266192 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6153
I20260809 07:54:43.273151 6429 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:54:43.277400 6319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:43.277719 6429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: Bootstrap starting.
I20260809 07:54:43.296440 6429 log.cc:826] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: Log is configured to *not* fsync() on all Append() calls
I20260809 07:54:43.306778 6429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:54:43.307361 6429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: Bootstrap complete.
I20260809 07:54:43.314921 6429 raft_consensus.cc:359] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } }
I20260809 07:54:43.315618 6429 raft_consensus.cc:740] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24efeb90e8c4810b0e68d760a623266, State: Initialized, Role: FOLLOWER
I20260809 07:54:43.316319 6429 consensus_queue.cc:260] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } }
I20260809 07:54:43.317327 6435 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:43.317670 6435 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:43.318456 6429 sys_catalog.cc:565] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:54:43.329576 6448 catalog_manager.cc:1295] Loaded cluster ID: 1cc4c017759e4c39a1884a11ea9fe0dc
I20260809 07:54:43.329766 6448 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: loading cluster ID for follower catalog manager: success
I20260809 07:54:43.337217 6448 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: acquiring CA information for follower catalog manager: success
I20260809 07:54:43.341768 6448 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:54:43.388209 6036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:43.388602 6036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:43.396612 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6153
I20260809 07:54:43.400108 6435 sys_catalog.cc:455] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e8f893dde3c49099e2f7d5c7a321b3c. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:43.400380 6435 sys_catalog.cc:458] T 00000000000000000000000000000000 P f24efeb90e8c4810b0e68d760a623266 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:54:43.425009 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:42449
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--webserver_port=36423
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783 with env {}
W20260809 07:54:43.829429 6461 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:43.829756 6461 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:43.829905 6461 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:43.838369 6461 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:43.838553 6461 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:43.848348 6461 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:42449
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=36423
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:43.849987 6461 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:43.851913 6461 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:43.862236 6467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:43.862296 6470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:43.862389 6466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:43.863430 6461 server_base.cc:1061] running on GCE node
I20260809 07:54:43.864120 6461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:43.865446 6461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:43.866693 6461 hybrid_clock.cc:648] HybridClock initialized: now 1786262083866565 us; error 92 us; skew 500 ppm
I20260809 07:54:43.867026 6461 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:43.869091 6461 webserver.cc:533] Webserver started at http://127.26.37.1:36423/ using document root <none> and password file <none>
I20260809 07:54:43.869937 6461 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:43.870081 6461 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:43.875548 6461 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:54:43.878823 6477 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:43.880187 6461 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:54:43.880366 6461 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "09cd829aea0b4ff0b349c901223449bc"
format_stamp: "Formatted at 2026-08-09 07:54:40 on dist-test-slave-bw88"
I20260809 07:54:43.881086 6461 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:43.892967 6036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f24efeb90e8c4810b0e68d760a623266" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39799 } } peers { permanent_uuid: "a53948f086e84784a6d898a4e5eea0a9" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 39407 } } peers { permanent_uuid: "1e8f893dde3c49099e2f7d5c7a321b3c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "8f640506aeb24d8ba3741e393485184e" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 39783 } attrs { promote: false } } }
I20260809 07:54:43.893216 6036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c [sys.catalog]: This master's current role is: LEADER
I20260809 07:54:43.897280 6461 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:43.898151 6461 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:43.898401 6461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:43.899302 6461 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:43.900950 6461 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:43.901028 6461 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:43.901114 6461 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:43.901173 6461 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:43.935129 6461 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:42449
I20260809 07:54:43.935185 6624 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:42449 every 8 connection(s)
I20260809 07:54:43.936714 6461 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:43.938752 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6461
I20260809 07:54:43.939155 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6461
I20260809 07:54:43.976886 6625 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46289
I20260809 07:54:43.977162 6628 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39407
I20260809 07:54:43.977384 6625 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:43.977437 6628 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:43.978230 6626 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39799
I20260809 07:54:43.978400 6626 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:43.978684 6625 heartbeater.cc:507] Master 127.26.37.60:46289 requested a full tablet report, sending...
I20260809 07:54:43.978721 6628 heartbeater.cc:507] Master 127.26.37.61:39407 requested a full tablet report, sending...
I20260809 07:54:43.978871 6626 heartbeater.cc:507] Master 127.26.37.62:39799 requested a full tablet report, sending...
I20260809 07:54:43.979228 6627 heartbeater.cc:344] Connected to a master server at 127.26.37.59:39783
I20260809 07:54:43.979413 6618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:43.979416 6627 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:43.981034 5487 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:43.981606 5582 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:43.981982 6386 ts_manager.cc:194] Registered new tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:43.982529 5582 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:60639
I20260809 07:54:44.073105 6627 heartbeater.cc:507] Master 127.26.37.59:39783 requested a full tablet report, sending...
I20260809 07:54:44.074038 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6461
I20260809 07:54:44.074959 6087 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
W20260809 07:54:44.098905 6070 connection.cc:573] server connection from 127.26.37.1:59753 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:44.099990 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:42449
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--webserver_port=36423
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783
--fault_before_start=1.0 with env {}
W20260809 07:54:44.469836 6643 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:44.470158 6643 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:44.470249 6643 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:54:44.470307 6643 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:44.478865 6643 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:44.479166 6643 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:44.489454 6643 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:42449
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=36423
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:44.491101 6643 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:44.492950 6643 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:44.502933 6648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:44.503069 6649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:44.503645 6652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:44.504945 6643 server_base.cc:1061] running on GCE node
I20260809 07:54:44.505543 6643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:44.506646 6643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:44.507896 6643 hybrid_clock.cc:648] HybridClock initialized: now 1786262084507708 us; error 113 us; skew 500 ppm
I20260809 07:54:44.508208 6643 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:44.510411 6643 webserver.cc:533] Webserver started at http://127.26.37.1:36423/ using document root <none> and password file <none>
I20260809 07:54:44.511173 6643 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:44.511332 6643 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:44.516892 6643 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 07:54:44.519773 6658 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:44.521121 6643 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:54:44.521391 6643 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "09cd829aea0b4ff0b349c901223449bc"
format_stamp: "Formatted at 2026-08-09 07:54:40 on dist-test-slave-bw88"
I20260809 07:54:44.522210 6643 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:44.557694 6643 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:44.558626 6643 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:44.558930 6643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:44.559945 6643 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:44.561749 6643 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:44.561885 6643 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:44.562034 6643 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:44.562127 6643 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:54:44.562583 6643 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:54:44.562733 6643 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:54:44.562810 6643 fault_injection.cc:59] injecting fault for kudu (pid 6643): FLAGS_fault_before_start (process will exit)
I20260809 07:54:44.587000 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:42449
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289
--webserver_port=36423
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783 with env {}
W20260809 07:54:44.950148 6668 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:54:44.950538 6668 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:54:44.950721 6668 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:54:44.959167 6668 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:54:44.959375 6668 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:54:44.969616 6668 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44675,127.26.37.21:44161,127.26.37.22:34047,127.26.37.23:44401,127.26.37.24:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:42449
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=36423
--tserver_master_addrs=127.26.37.62:39799,127.26.37.61:39407,127.26.37.60:46289,127.26.37.59:39783
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:54:44.971312 6668 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:54:44.973241 6668 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:54:44.983398 6674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:44.983675 6675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:54:44.983316 6677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:54:44.984407 6668 server_base.cc:1061] running on GCE node
I20260809 07:54:44.985062 6668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:54:44.986251 6668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:54:44.987534 6668 hybrid_clock.cc:648] HybridClock initialized: now 1786262084987374 us; error 127 us; skew 500 ppm
I20260809 07:54:44.987926 6668 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:54:44.990168 6668 webserver.cc:533] Webserver started at http://127.26.37.1:36423/ using document root <none> and password file <none>
I20260809 07:54:44.990965 6668 fs_manager.cc:362] Metadata directory not provided
I20260809 07:54:44.991093 6668 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:54:44.996570 6668 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:54:44.999404 6684 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:45.000856 6668 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:54:45.001031 6668 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "09cd829aea0b4ff0b349c901223449bc"
format_stamp: "Formatted at 2026-08-09 07:54:40 on dist-test-slave-bw88"
I20260809 07:54:45.002031 6668 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:54:45.014969 6668 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:54:45.015759 6668 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:54:45.016038 6668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:54:45.017077 6668 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:54:45.019243 6668 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:54:45.019416 6668 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 07:54:45.019562 6668 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:54:45.019627 6668 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:54:45.058193 6668 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:42449
I20260809 07:54:45.058229 6827 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:42449 every 8 connection(s)
I20260809 07:54:45.059897 6668 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:54:45.063392 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 6668
I20260809 07:54:45.063709 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6668
I20260809 07:54:45.093225 6832 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39799
I20260809 07:54:45.093735 6832 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:45.094981 6832 heartbeater.cc:507] Master 127.26.37.62:39799 requested a full tablet report, sending...
I20260809 07:54:45.095243 6834 heartbeater.cc:344] Connected to a master server at 127.26.37.61:39407
I20260809 07:54:45.095429 6834 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:45.096053 6834 heartbeater.cc:507] Master 127.26.37.61:39407 requested a full tablet report, sending...
I20260809 07:54:45.097118 5487 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:45.097281 6386 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:45.099481 6833 heartbeater.cc:344] Connected to a master server at 127.26.37.59:39783
I20260809 07:54:45.099637 6833 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:45.100097 6833 heartbeater.cc:507] Master 127.26.37.59:39783 requested a full tablet report, sending...
I20260809 07:54:45.101369 6087 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:45.101953 6831 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46289
I20260809 07:54:45.102068 6831 heartbeater.cc:461] Registering TS with master...
I20260809 07:54:45.102586 6831 heartbeater.cc:507] Master 127.26.37.60:46289 requested a full tablet report, sending...
I20260809 07:54:45.103664 5582 ts_manager.cc:194] Re-registered known tserver with Master: 09cd829aea0b4ff0b349c901223449bc (127.26.37.1:42449)
I20260809 07:54:45.104514 5582 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:39711
I20260809 07:54:45.105649 6823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:45.198355 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6668
I20260809 07:54:45.227679 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 6343
I20260809 07:54:45.234730 6424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:45.338788 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 6343
W20260809 07:54:45.375757 5566 connection.cc:573] client connection to 127.26.37.62:39799 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:54:45.375916 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5442
I20260809 07:54:45.377415 5524 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 07:54:45.382690 5566 consensus_peers.cc:597] T 00000000000000000000000000000000 P 1e8f893dde3c49099e2f7d5c7a321b3c -> Peer f24efeb90e8c4810b0e68d760a623266 (127.26.37.62:39799): Couldn't send request to peer f24efeb90e8c4810b0e68d760a623266. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:39799: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:54:45.519266 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5442
I20260809 07:54:45.546042 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5536
I20260809 07:54:45.547298 5622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:45.660197 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5536
I20260809 07:54:45.688009 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 5904
I20260809 07:54:45.689229 6124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:54:45.799813 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 5904
2026-08-09T07:54:45Z chronyd exiting
2026-08-09T07:54:45Z chronyd exiting
2026-08-09T07:54:45Z chronyd exiting
2026-08-09T07:54:45Z chronyd exiting
2026-08-09T07:54:45Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/2 (7603 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/3
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:54:53: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:55:01.991405 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34629
--webserver_interface=127.26.37.62
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925 with env {}
W20260809 07:55:02.340466 7213 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:02.340816 7213 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:02.340947 7213 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:02.348919 7213 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:02.348999 7213 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:02.349104 7213 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:02.349196 7213 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:02.358598 7213 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:02.360126 7213 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:02.361979 7213 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:02.370110 7222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:02.370235 7226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:02.371431 7230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:02.372047 7213 server_base.cc:1061] running on GCE node
I20260809 07:55:02.372834 7213 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:02.373034 7213 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:02.373133 7213 hybrid_clock.cc:648] HybridClock initialized: now 1786262102373092 us; error 0 us; skew 500 ppm
I20260809 07:55:02.373551 7213 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:02.376317 7213 webserver.cc:533] Webserver started at http://127.26.37.62:40183/ using document root <none> and password file <none>
I20260809 07:55:02.377082 7213 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:02.377210 7213 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:02.377615 7213 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:02.381314 7213 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "5a49f61c729f4bf5babc869212715595"
format_stamp: "Formatted at 2026-08-09 07:55:02 on dist-test-slave-bw88"
I20260809 07:55:02.382237 7213 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "5a49f61c729f4bf5babc869212715595"
format_stamp: "Formatted at 2026-08-09 07:55:02 on dist-test-slave-bw88"
I20260809 07:55:02.387677 7213 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 07:55:02.391002 7238 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:02.392288 7213 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:55:02.392486 7213 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "5a49f61c729f4bf5babc869212715595"
format_stamp: "Formatted at 2026-08-09 07:55:02 on dist-test-slave-bw88"
I20260809 07:55:02.392731 7213 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:02.416695 7213 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:02.417711 7213 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:02.418018 7213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:02.438645 7213 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34629
I20260809 07:55:02.438665 7298 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34629 every 8 connection(s)
I20260809 07:55:02.440223 7213 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:55:02.444613 7299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:02.445159 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7213
I20260809 07:55:02.445395 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:55:02.446682 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:33247
--webserver_interface=127.26.37.61
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925 with env {}
I20260809 07:55:02.453269 7299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:02.474383 7299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:02.475797 7240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:33247: connect: Connection refused (error 111)
W20260809 07:55:02.477470 7299 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:33247: Network error: Client connection negotiation failed: client connection to 127.26.37.61:33247: connect: Connection refused (error 111)
I20260809 07:55:02.526849 7299 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } attempt: 1
W20260809 07:55:02.528970 7299 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:33247: Network error: Client connection negotiation failed: client connection to 127.26.37.61:33247: connect: Connection refused (error 111)
I20260809 07:55:02.597254 7299 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } attempt: 2
W20260809 07:55:02.599499 7299 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:33247: Network error: Client connection negotiation failed: client connection to 127.26.37.61:33247: connect: Connection refused (error 111)
I20260809 07:55:02.690783 7299 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } attempt: 3
W20260809 07:55:02.692940 7299 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:33247: Network error: Client connection negotiation failed: client connection to 127.26.37.61:33247: connect: Connection refused (error 111)
W20260809 07:55:02.809664 7304 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:02.810022 7304 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:02.810168 7304 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:02.818351 7304 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:02.818436 7304 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:02.818568 7304 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:02.818653 7304 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:02.828361 7304 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:33247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:02.830425 7304 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:02.832345 7304 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:55:02.836196 7299 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } attempt: 4
W20260809 07:55:02.838493 7299 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:33247: Network error: Client connection negotiation failed: client connection to 127.26.37.61:33247: connect: Connection refused (error 111)
W20260809 07:55:02.840925 7313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:02.840891 7311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:02.842007 7315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:02.842832 7304 server_base.cc:1061] running on GCE node
I20260809 07:55:02.843585 7304 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:02.843786 7304 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:02.843889 7304 hybrid_clock.cc:648] HybridClock initialized: now 1786262102843867 us; error 0 us; skew 500 ppm
I20260809 07:55:02.844242 7304 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:02.846997 7304 webserver.cc:533] Webserver started at http://127.26.37.61:46313/ using document root <none> and password file <none>
I20260809 07:55:02.847750 7304 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:02.847890 7304 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:02.848275 7304 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:02.850689 7304 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "bb74ca8d446c458599fc49b81a6c3bcd"
format_stamp: "Formatted at 2026-08-09 07:55:02 on dist-test-slave-bw88"
I20260809 07:55:02.851531 7304 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "bb74ca8d446c458599fc49b81a6c3bcd"
format_stamp: "Formatted at 2026-08-09 07:55:02 on dist-test-slave-bw88"
I20260809 07:55:02.856711 7304 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:55:02.860095 7321 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:02.861392 7304 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:55:02.861591 7304 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "bb74ca8d446c458599fc49b81a6c3bcd"
format_stamp: "Formatted at 2026-08-09 07:55:02 on dist-test-slave-bw88"
I20260809 07:55:02.861804 7304 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:02.882267 7304 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:02.883164 7304 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:02.883453 7304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:02.904155 7304 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:33247
I20260809 07:55:02.904181 7393 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:33247 every 8 connection(s)
I20260809 07:55:02.905737 7304 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:55:02.906289 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7304
I20260809 07:55:02.906513 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:55:02.907704 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:41925
--webserver_interface=127.26.37.60
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925 with env {}
I20260809 07:55:02.910415 7394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:02.916201 7394 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:02.929265 7394 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:02.938745 7394 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:02.939960 7326 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
W20260809 07:55:02.941697 7394 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41925: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
I20260809 07:55:02.991039 7394 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } attempt: 1
W20260809 07:55:02.993187 7394 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41925: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
I20260809 07:55:03.061473 7394 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } attempt: 2
W20260809 07:55:03.063851 7394 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41925: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
I20260809 07:55:03.137787 7299 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } attempt: 5
I20260809 07:55:03.145295 7299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:03.147553 7299 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41925: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
I20260809 07:55:03.155053 7394 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } attempt: 3
W20260809 07:55:03.156795 7394 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41925: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
I20260809 07:55:03.198798 7299 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } attempt: 1
W20260809 07:55:03.200884 7299 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41925: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
I20260809 07:55:03.269134 7299 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } attempt: 2
W20260809 07:55:03.271307 7299 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41925: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
W20260809 07:55:03.273315 7399 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:03.273701 7399 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:03.273787 7399 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:03.281970 7399 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:03.282055 7399 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:03.282111 7399 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:03.282160 7399 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:03.292035 7399 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:41925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:03.293645 7399 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:03.295411 7399 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:55:03.300043 7394 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } attempt: 4
W20260809 07:55:03.303587 7408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:03.303550 7407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:03.305244 7394 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41925: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41925: connect: Connection refused (error 111)
W20260809 07:55:03.305884 7411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:03.306579 7399 server_base.cc:1061] running on GCE node
I20260809 07:55:03.307396 7399 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:03.307590 7399 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:03.307669 7399 hybrid_clock.cc:648] HybridClock initialized: now 1786262103307652 us; error 0 us; skew 500 ppm
I20260809 07:55:03.308037 7399 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:03.310843 7399 webserver.cc:533] Webserver started at http://127.26.37.60:40521/ using document root <none> and password file <none>
I20260809 07:55:03.311612 7399 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:03.311743 7399 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:03.312161 7399 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:03.314735 7399 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "79a0d69c482349298d8330b6de948fa4"
format_stamp: "Formatted at 2026-08-09 07:55:03 on dist-test-slave-bw88"
I20260809 07:55:03.315594 7399 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "79a0d69c482349298d8330b6de948fa4"
format_stamp: "Formatted at 2026-08-09 07:55:03 on dist-test-slave-bw88"
I20260809 07:55:03.321171 7399 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 07:55:03.324610 7419 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:03.326025 7399 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:55:03.326218 7399 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "79a0d69c482349298d8330b6de948fa4"
format_stamp: "Formatted at 2026-08-09 07:55:03 on dist-test-slave-bw88"
I20260809 07:55:03.326428 7399 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:03.338245 7399 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:03.339131 7399 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:03.339430 7399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:03.360152 7399 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:41925
I20260809 07:55:03.360191 7487 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:41925 every 8 connection(s)
I20260809 07:55:03.361828 7399 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:55:03.364503 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7399
I20260809 07:55:03.364715 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:55:03.366551 7488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:03.374969 7488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:03.377683 7299 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } attempt: 3
I20260809 07:55:03.388947 7488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:03.395665 7488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:03.397645 7299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Bootstrap starting.
I20260809 07:55:03.401794 7299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:03.402930 7299 log.cc:826] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:03.406075 7299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: No bootstrap required, opened a new log
I20260809 07:55:03.406322 7488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4: Bootstrap starting.
I20260809 07:55:03.409133 7488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:03.410210 7488 log.cc:826] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:03.411545 7299 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
I20260809 07:55:03.411787 7299 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:03.411932 7299 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a49f61c729f4bf5babc869212715595, State: Initialized, Role: FOLLOWER
I20260809 07:55:03.412493 7299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
I20260809 07:55:03.412889 7488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4: No bootstrap required, opened a new log
I20260809 07:55:03.414810 7299 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:03.415017 7500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.415337 7500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:03.419080 7488 raft_consensus.cc:359] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
I20260809 07:55:03.419354 7488 raft_consensus.cc:385] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:03.419497 7488 raft_consensus.cc:740] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79a0d69c482349298d8330b6de948fa4, State: Initialized, Role: FOLLOWER
I20260809 07:55:03.420238 7488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
W20260809 07:55:03.420737 7244 tablet.cc:2406] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:55:03.424526 7213 master_runner.cc:187] Error getting master registration for 127.26.37.61:33247: OK, instance_id { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" instance_seqno: 1786262102898593 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:55:03.429399 7213 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:55:03.431365 7514 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:03.431495 7514 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:03.432170 7515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.432499 7515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:03.433467 7488 sys_catalog.cc:565] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:03.448858 7399 master_runner.cc:187] Error getting master registration for 127.26.37.61:33247: OK, instance_id { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" instance_seqno: 1786262102898593 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:55:03.457551 7399 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:55:03.455880 7528 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:03.457729 7528 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:03.477948 7500 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:55:03.478255 7500 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
I20260809 07:55:03.480132 7500 leader_election.cc:290] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb74ca8d446c458599fc49b81a6c3bcd (127.26.37.61:33247), 79a0d69c482349298d8330b6de948fa4 (127.26.37.60:41925)
I20260809 07:55:03.481168 7453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a49f61c729f4bf5babc869212715595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a0d69c482349298d8330b6de948fa4" is_pre_election: true
I20260809 07:55:03.481725 7453 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a49f61c729f4bf5babc869212715595 in term 0.
I20260809 07:55:03.482472 7242 leader_election.cc:304] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a49f61c729f4bf5babc869212715595, 79a0d69c482349298d8330b6de948fa4; no voters:
I20260809 07:55:03.485510 7500 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:55:03.485689 7500 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:55:03.485809 7500 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:55:03.489254 7500 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
I20260809 07:55:03.490034 7500 leader_election.cc:290] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [CANDIDATE]: Term 1 election: Requested vote from peers bb74ca8d446c458599fc49b81a6c3bcd (127.26.37.61:33247), 79a0d69c482349298d8330b6de948fa4 (127.26.37.60:41925)
I20260809 07:55:03.491025 7453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a49f61c729f4bf5babc869212715595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a0d69c482349298d8330b6de948fa4"
I20260809 07:55:03.491221 7453 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:55:03.494719 7453 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a49f61c729f4bf5babc869212715595 in term 1.
I20260809 07:55:03.495215 7242 leader_election.cc:304] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a49f61c729f4bf5babc869212715595, 79a0d69c482349298d8330b6de948fa4; no voters:
I20260809 07:55:03.495604 7500 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:55:03.505522 7500 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 1 LEADER]: Becoming Leader. State: Replica: 5a49f61c729f4bf5babc869212715595, State: Running, Role: LEADER
I20260809 07:55:03.508226 7500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
I20260809 07:55:03.514169 7530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a49f61c729f4bf5babc869212715595. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.514672 7530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:03.521715 7532 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:55:03.524274 7532 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:55:03.535769 7453 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 1 FOLLOWER]: Refusing update from remote peer 5a49f61c729f4bf5babc869212715595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:55:03.536803 7500 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:55:03.559306 7515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a49f61c729f4bf5babc869212715595. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.559597 7515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:03.564885 7530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.565171 7530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:03.567417 7515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.567631 7515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:03.571604 7532 catalog_manager.cc:1383] Generated new cluster ID: f8a0cd79683b4219a6293b7edca6ee76
I20260809 07:55:03.571724 7532 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:55:03.592981 7532 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:55:03.594786 7532 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:55:03.604488 7394 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } attempt: 5
I20260809 07:55:03.618371 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: Bootstrap starting.
I20260809 07:55:03.626631 7394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:03.628094 7394 log.cc:826] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:03.628718 7532 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Generated new TSK 0
I20260809 07:55:03.629817 7532 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:55:03.633947 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: No bootstrap required, opened a new log
I20260809 07:55:03.642536 7394 raft_consensus.cc:359] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
I20260809 07:55:03.642913 7394 raft_consensus.cc:385] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:03.643100 7394 raft_consensus.cc:740] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb74ca8d446c458599fc49b81a6c3bcd, State: Initialized, Role: FOLLOWER
I20260809 07:55:03.643934 7394 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } }
I20260809 07:55:03.649507 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--time_source=system_unsync with env {}
I20260809 07:55:03.676893 7543 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.678211 7543 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:03.676932 7394 sys_catalog.cc:565] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:03.677016 7354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a49f61c729f4bf5babc869212715595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb74ca8d446c458599fc49b81a6c3bcd"
I20260809 07:55:03.677057 7353 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:55:03.676952 7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a49f61c729f4bf5babc869212715595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" is_pre_election: true
I20260809 07:55:03.687991 7353 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 1 FOLLOWER]: Refusing update from remote peer 5a49f61c729f4bf5babc869212715595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:55:03.689083 7530 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:55:03.741360 7543 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a49f61c729f4bf5babc869212715595. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.741742 7543 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:03.742192 7553 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:03.742573 7553 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:03.759054 7569 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 07:55:04.107703 7555 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:04.108035 7555 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:04.108165 7555 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:04.116258 7555 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:04.116444 7555 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:04.127074 7555 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:04.128602 7555 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:04.130353 7555 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:04.139766 7575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:04.139797 7574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:04.141034 7579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:04.141993 7555 server_base.cc:1061] running on GCE node
I20260809 07:55:04.142529 7555 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:04.142668 7555 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:04.142731 7555 hybrid_clock.cc:648] HybridClock initialized: now 1786262104142714 us; error 0 us; skew 500 ppm
I20260809 07:55:04.142968 7555 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:04.145150 7555 webserver.cc:533] Webserver started at http://127.26.37.1:33659/ using document root <none> and password file <none>
I20260809 07:55:04.146099 7555 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:04.146232 7555 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:04.146610 7555 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:04.149019 7555 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "94c054e210b84f2da1dc085da7055cbc"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:04.149816 7555 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "94c054e210b84f2da1dc085da7055cbc"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:04.155202 7555 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:55:04.158468 7589 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:04.159793 7555 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 07:55:04.159973 7555 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "94c054e210b84f2da1dc085da7055cbc"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:04.160184 7555 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:04.175338 7555 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:04.176221 7555 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:04.176517 7555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:04.177515 7555 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:04.179172 7555 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:04.179301 7555 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:04.179450 7555 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:04.179540 7555 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:04.215181 7555 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35181
I20260809 07:55:04.215240 7720 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35181 every 8 connection(s)
I20260809 07:55:04.216764 7555 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:04.224818 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7555
I20260809 07:55:04.225047 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:55:04.248468 7530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } }
I20260809 07:55:04.248816 7530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:04.259428 7723 heartbeater.cc:344] Connected to a master server at 127.26.37.61:33247
I20260809 07:55:04.259828 7723 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:04.259474 7721 heartbeater.cc:344] Connected to a master server at 127.26.37.60:41925
I20260809 07:55:04.260869 7721 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:04.261070 7723 heartbeater.cc:507] Master 127.26.37.61:33247 requested a full tablet report, sending...
I20260809 07:55:04.261550 7721 heartbeater.cc:507] Master 127.26.37.60:41925 requested a full tablet report, sending...
I20260809 07:55:04.262646 7722 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34629
I20260809 07:55:04.262789 7722 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:04.263298 7722 heartbeater.cc:507] Master 127.26.37.62:34629 requested a full tablet report, sending...
I20260809 07:55:04.263692 7342 ts_manager.cc:194] Registered new tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:04.263854 7442 ts_manager.cc:194] Registered new tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:04.264916 7258 ts_manager.cc:194] Registered new tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:04.266762 7258 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:53837
I20260809 07:55:04.271193 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:04.273710 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:43055
--webserver_interface=127.26.37.59
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055 with env {}
I20260809 07:55:04.459585 7528 catalog_manager.cc:1295] Loaded cluster ID: f8a0cd79683b4219a6293b7edca6ee76
I20260809 07:55:04.459674 7528 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4: loading cluster ID for follower catalog manager: success
I20260809 07:55:04.462379 7528 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4: acquiring CA information for follower catalog manager: success
I20260809 07:55:04.464085 7528 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:55:04.630028 7735 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:04.630383 7735 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:04.630519 7735 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:04.638732 7735 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:04.638852 7735 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:04.638983 7735 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:04.639056 7735 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:04.648424 7735 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:43055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:04.650135 7735 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:04.651907 7735 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:04.660288 7742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:04.660255 7744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:04.661077 7746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:04.661728 7735 server_base.cc:1061] running on GCE node
I20260809 07:55:04.662415 7735 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:04.662596 7735 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:04.662703 7735 hybrid_clock.cc:648] HybridClock initialized: now 1786262104662665 us; error 0 us; skew 500 ppm
I20260809 07:55:04.663054 7735 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:04.665791 7735 webserver.cc:533] Webserver started at http://127.26.37.59:40389/ using document root <none> and password file <none>
I20260809 07:55:04.666509 7735 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:04.666644 7735 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:04.667068 7735 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:04.669480 7735 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "2a009e76a84f45a6b47befbc25cbbd84"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:04.670190 7735 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "2a009e76a84f45a6b47befbc25cbbd84"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:04.675285 7735 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:55:04.678602 7753 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:04.679847 7735 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:55:04.680029 7735 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "2a009e76a84f45a6b47befbc25cbbd84"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:04.680248 7735 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:04.701140 7735 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:04.702015 7735 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:04.702297 7735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:04.723904 7735 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:43055
I20260809 07:55:04.723938 7818 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:43055 every 8 connection(s)
I20260809 07:55:04.725440 7735 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:55:04.728425 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7735
I20260809 07:55:04.730065 7821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:04.737548 7821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:04.747423 7821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:04.753482 7821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:04.759227 7821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:04.764851 7569 catalog_manager.cc:1295] Loaded cluster ID: f8a0cd79683b4219a6293b7edca6ee76
I20260809 07:55:04.764997 7569 catalog_manager.cc:1588] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: loading cluster ID for follower catalog manager: success
I20260809 07:55:04.768815 7821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Bootstrap starting.
I20260809 07:55:04.768882 7569 catalog_manager.cc:1610] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: acquiring CA information for follower catalog manager: success
I20260809 07:55:04.771410 7569 catalog_manager.cc:1638] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:55:04.771785 7821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:04.772848 7821 log.cc:826] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:04.775691 7821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: No bootstrap required, opened a new log
I20260809 07:55:04.780640 7821 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } }
I20260809 07:55:04.780936 7821 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:04.781042 7821 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a009e76a84f45a6b47befbc25cbbd84, State: Initialized, Role: FOLLOWER
I20260809 07:55:04.781692 7821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } }
I20260809 07:55:04.782661 7832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } } }
I20260809 07:55:04.783023 7832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:04.783774 7821 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:55:04.794168 7848 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:04.794305 7848 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:04.796000 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7555
I20260809 07:55:04.803459 7735 master_runner.cc:429] Detected that this master 2a009e76a84f45a6b47befbc25cbbd84 is joining an existing cluster
I20260809 07:55:04.803579 7735 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:55:04.804589 7735 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:55:04.804977 7735 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:55:04.805238 7735 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: stopping tablet replica
I20260809 07:55:04.807301 7735 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:43055
I20260809 07:55:04.817231 7716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:04.933003 7258 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:43055: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 43055
}
attrs {
promote: true
}
}
dest_uuid: "5a49f61c729f4bf5babc869212715595"
cas_config_opid_index: -1
I20260809 07:55:04.934947 7258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
W20260809 07:55:04.948066 7242 consensus_peers.cc:597] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 -> Peer 2a009e76a84f45a6b47befbc25cbbd84 (127.26.37.59:43055): Couldn't send request to peer 2a009e76a84f45a6b47befbc25cbbd84. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:55:04.949483 7357 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 1 FOLLOWER]: Refusing update from remote peer 5a49f61c729f4bf5babc869212715595: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:55:04.950248 7732 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:55:04.957687 7453 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 1 FOLLOWER]: Refusing update from remote peer 5a49f61c729f4bf5babc869212715595: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:55:04.958446 7732 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:55:04.988613 7873 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2a009e76a84f45a6b47befbc25cbbd84 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:04.993023 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7555
I20260809 07:55:04.999118 7873 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:43055
I20260809 07:55:04.999784 7872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:05.000054 7872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:04.999542 7357 raft_consensus.cc:2955] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2a009e76a84f45a6b47befbc25cbbd84 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:05.000656 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:05.000906 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:05.001260 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:05.001485 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:05.004635 7735 master.cc:562] Master@127.26.37.59:43055 shutting down...
I20260809 07:55:05.008774 7453 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2a009e76a84f45a6b47befbc25cbbd84 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:05.029671 7880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:05.030022 7880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:05.030660 7879 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a49f61c729f4bf5babc869212715595" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:05.031037 7879 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:05.037411 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35181
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--webserver_port=33659
--webserver_interface=127.26.37.1
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055 with env {}
I20260809 07:55:05.040212 7735 master.cc:584] Master@127.26.37.59:43055 shutdown complete.
I20260809 07:55:05.040306 7735 master_runner.cc:306] Clearing existing system tablet
I20260809 07:55:05.040722 7735 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:05.040832 7735 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:05.054188 7735 fs_manager.cc:714] Time spent opening directory manager: real 0.012s user 0.003s sys 0.009s
I20260809 07:55:05.069697 7884 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:05.072203 7735 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:05.072633 7735 fs_manager.cc:730] Time spent opening block manager: real 0.017s user 0.004s sys 0.008s
I20260809 07:55:05.072789 7735 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "2a009e76a84f45a6b47befbc25cbbd84"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:05.077652 7735 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:55:05.084821 7735 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:55:05.085094 7735 log.cc:1199] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:55:05.085816 7735 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Deleting consensus metadata
I20260809 07:55:05.086148 7735 master_runner.cc:316] Copying system tablet from 127.26.37.62:34629
I20260809 07:55:05.145028 7735 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.62:34629
I20260809 07:55:05.168045 7279 tablet_copy_service.cc:140] P 5a49f61c729f4bf5babc869212715595: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2a009e76a84f45a6b47befbc25cbbd84 ({username='slave'} at 127.0.0.1:58874)
I20260809 07:55:05.168390 7279 tablet_copy_service.cc:161] P 5a49f61c729f4bf5babc869212715595: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2a009e76a84f45a6b47befbc25cbbd84 at {username='slave'} at 127.0.0.1:58874: session id = 2a009e76a84f45a6b47befbc25cbbd84-00000000000000000000000000000000
I20260809 07:55:05.173055 7279 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:55:05.175843 7735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:55:05.184285 7735 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: tablet copy: Starting download of 0 data blocks...
I20260809 07:55:05.184626 7735 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: tablet copy: Starting download of 1 WAL segments...
I20260809 07:55:05.187721 7735 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:55:05.192390 7279 tablet_copy_service.cc:342] P 5a49f61c729f4bf5babc869212715595: Request end of tablet copy session 2a009e76a84f45a6b47befbc25cbbd84-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:58874
I20260809 07:55:05.192651 7279 tablet_copy_service.cc:434] P 5a49f61c729f4bf5babc869212715595: ending tablet copy session 2a009e76a84f45a6b47befbc25cbbd84-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2a009e76a84f45a6b47befbc25cbbd84
I20260809 07:55:05.207966 7735 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:05.221792 7892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:05.223932 7894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:05.234577 7897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:05.235128 7735 server_base.cc:1061] running on GCE node
I20260809 07:55:05.235622 7735 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:05.235760 7735 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:05.235829 7735 hybrid_clock.cc:648] HybridClock initialized: now 1786262105235806 us; error 0 us; skew 500 ppm
I20260809 07:55:05.236089 7735 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:05.252699 7735 webserver.cc:533] Webserver started at http://127.26.37.59:43955/ using document root <none> and password file <none>
I20260809 07:55:05.253180 7735 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:05.253279 7735 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:05.262655 7735 fs_manager.cc:714] Time spent opening directory manager: real 0.008s user 0.004s sys 0.004s
I20260809 07:55:05.280818 7902 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:05.281425 7735 fs_manager.cc:730] Time spent opening block manager: real 0.018s user 0.001s sys 0.009s
I20260809 07:55:05.281615 7735 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "2a009e76a84f45a6b47befbc25cbbd84"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:05.281814 7735 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:05.331038 7735 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:05.331979 7735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:05.516984 7735 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:43055
I20260809 07:55:05.517089 7969 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:43055 every 8 connection(s)
I20260809 07:55:05.518265 7735 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:55:05.524084 7970 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:55:05.528178 7970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Bootstrap starting.
I20260809 07:55:05.547451 7970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:55:05.548269 7970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: Bootstrap complete.
I20260809 07:55:05.549618 7970 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
I20260809 07:55:05.550052 7970 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2a009e76a84f45a6b47befbc25cbbd84, State: Initialized, Role: LEARNER
I20260809 07:55:05.550419 7970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
I20260809 07:55:05.552910 7974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:05.553288 7974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:05.553433 7970 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:05.569734 7992 catalog_manager.cc:1295] Loaded cluster ID: f8a0cd79683b4219a6293b7edca6ee76
I20260809 07:55:05.569864 7992 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: loading cluster ID for follower catalog manager: success
I20260809 07:55:05.572742 7992 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: acquiring CA information for follower catalog manager: success
I20260809 07:55:05.575024 7992 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:55:05.591285 7883 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:05.591671 7883 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:05.591817 7883 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:05.601505 7883 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:05.601722 7883 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:05.611756 7883 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35181
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33659
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:05.613422 7883 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:05.615377 7883 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:05.625767 7999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:05.626525 7998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:05.626641 8001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:05.627104 7883 server_base.cc:1061] running on GCE node
I20260809 07:55:05.627648 7883 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:05.627790 7883 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:05.627851 7883 hybrid_clock.cc:648] HybridClock initialized: now 1786262105627835 us; error 0 us; skew 500 ppm
I20260809 07:55:05.628106 7883 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:05.630301 7883 webserver.cc:533] Webserver started at http://127.26.37.1:33659/ using document root <none> and password file <none>
I20260809 07:55:05.630992 7883 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:05.631120 7883 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:05.636729 7883 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:55:05.639806 8007 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:05.641218 7883 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:55:05.641427 7883 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "94c054e210b84f2da1dc085da7055cbc"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:05.642163 7883 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:05.663414 7883 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:05.664659 7883 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:05.665015 7883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:05.667812 7883 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:05.670059 7883 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:05.670161 7883 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:05.670356 7883 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:05.670450 7883 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:05.711680 8154 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35181 every 8 connection(s)
I20260809 07:55:05.712096 7883 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35181
I20260809 07:55:05.714355 7883 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:05.719740 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 7883
I20260809 07:55:05.748708 8155 heartbeater.cc:344] Connected to a master server at 127.26.37.59:43055
I20260809 07:55:05.749163 8155 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:05.751051 8155 heartbeater.cc:507] Master 127.26.37.59:43055 requested a full tablet report, sending...
I20260809 07:55:05.754835 7925 ts_manager.cc:194] Registered new tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:05.762971 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:05.764082 8157 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34629
I20260809 07:55:05.764242 8157 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:05.764818 8160 heartbeater.cc:344] Connected to a master server at 127.26.37.61:33247
I20260809 07:55:05.764874 8157 heartbeater.cc:507] Master 127.26.37.62:34629 requested a full tablet report, sending...
I20260809 07:55:05.764979 8160 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:05.765522 8160 heartbeater.cc:507] Master 127.26.37.61:33247 requested a full tablet report, sending...
I20260809 07:55:05.766366 7258 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:05.767498 7258 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:46229
I20260809 07:55:05.769275 7342 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:05.770990 8156 heartbeater.cc:344] Connected to a master server at 127.26.37.60:41925
I20260809 07:55:05.771158 8156 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:05.773016 8156 heartbeater.cc:507] Master 127.26.37.60:41925 requested a full tablet report, sending...
I20260809 07:55:05.774567 7442 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:05.787281 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:05.787493 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7213
I20260809 07:55:05.788537 7291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:05.901014 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7213
I20260809 07:55:05.929502 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34629
--webserver_interface=127.26.37.62
--webserver_port=40183
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055 with env {}
W20260809 07:55:06.287708 8176 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:06.288069 8176 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:06.288204 8176 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:06.296627 8176 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:06.296716 8176 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:06.296808 8176 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:06.296887 8176 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:06.306994 8176 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=40183
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:06.308722 8176 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:06.310613 8176 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:06.318856 8181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:06.319430 8182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:06.319533 8186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:06.320310 8176 server_base.cc:1061] running on GCE node
I20260809 07:55:06.320976 8176 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:06.321131 8176 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:06.321226 8176 hybrid_clock.cc:648] HybridClock initialized: now 1786262106321190 us; error 0 us; skew 500 ppm
I20260809 07:55:06.321571 8176 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:06.324131 8176 webserver.cc:533] Webserver started at http://127.26.37.62:40183/ using document root <none> and password file <none>
I20260809 07:55:06.324812 8176 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:06.324915 8176 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:06.330132 8176 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:55:06.332863 8191 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:06.334244 8176 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:06.334432 8176 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "5a49f61c729f4bf5babc869212715595"
format_stamp: "Formatted at 2026-08-09 07:55:02 on dist-test-slave-bw88"
I20260809 07:55:06.335147 8176 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:06.351965 8176 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:06.352705 8176 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:06.352941 8176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:06.374680 8176 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34629
I20260809 07:55:06.374782 8261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34629 every 8 connection(s)
I20260809 07:55:06.376376 8176 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:55:06.383601 8263 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:55:06.385257 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 8176
I20260809 07:55:06.385623 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7883
I20260809 07:55:06.387735 8263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Bootstrap starting.
I20260809 07:55:06.394311 8147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:06.410109 8263 log.cc:826] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:06.421610 8263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:55:06.422246 8263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: Bootstrap complete.
I20260809 07:55:06.432017 8263 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
I20260809 07:55:06.432893 8263 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a49f61c729f4bf5babc869212715595, State: Initialized, Role: FOLLOWER
I20260809 07:55:06.433722 8263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
I20260809 07:55:06.434697 8271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:06.435004 8271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:06.435773 8263 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:06.446658 8285 catalog_manager.cc:1295] Loaded cluster ID: f8a0cd79683b4219a6293b7edca6ee76
I20260809 07:55:06.446763 8285 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: loading cluster ID for follower catalog manager: success
I20260809 07:55:06.450714 8285 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: acquiring CA information for follower catalog manager: success
I20260809 07:55:06.453135 8285 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:55:06.502213 8176 master_runner.cc:264] No leader master found from master 5a49f61c729f4bf5babc869212715595
I20260809 07:55:06.502406 8176 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:55:06.514830 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7883
I20260809 07:55:06.543916 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35181
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--webserver_port=33659
--webserver_interface=127.26.37.1
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055 with env {}
W20260809 07:55:06.904034 8296 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:06.904331 8296 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:06.904405 8296 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:06.912443 8296 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:06.912600 8296 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:06.922449 8296 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35181
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33659
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:06.924043 8296 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:06.925972 8296 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:06.935659 8302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:06.935685 8301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:06.936709 8305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:06.937804 8296 server_base.cc:1061] running on GCE node
I20260809 07:55:06.938459 8296 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:06.938668 8296 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:06.938830 8296 hybrid_clock.cc:648] HybridClock initialized: now 1786262106938804 us; error 0 us; skew 500 ppm
I20260809 07:55:06.939277 8296 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:06.941830 8296 webserver.cc:533] Webserver started at http://127.26.37.1:33659/ using document root <none> and password file <none>
I20260809 07:55:06.942663 8296 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:06.942811 8296 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:06.948586 8296 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:55:06.951426 8311 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:06.952764 8296 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:55:06.952955 8296 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "94c054e210b84f2da1dc085da7055cbc"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:06.953789 8296 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:06.971204 8296 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:06.972245 8296 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:06.972577 8296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:06.973667 8296 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:06.975490 8296 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:06.975600 8296 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:06.975754 8296 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:06.975811 8296 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:07.012086 8296 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35181
I20260809 07:55:07.012123 8460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35181 every 8 connection(s)
I20260809 07:55:07.013681 8296 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:07.020879 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 8296
I20260809 07:55:07.038887 8479 raft_consensus.cc:493] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a49f61c729f4bf5babc869212715595)
I20260809 07:55:07.039331 8479 raft_consensus.cc:515] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
I20260809 07:55:07.042824 8461 heartbeater.cc:344] Connected to a master server at 127.26.37.59:43055
I20260809 07:55:07.043342 8461 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:07.044874 8461 heartbeater.cc:507] Master 127.26.37.59:43055 requested a full tablet report, sending...
I20260809 07:55:07.044920 8464 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34629
I20260809 07:55:07.045161 8464 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:07.045933 8464 heartbeater.cc:507] Master 127.26.37.62:34629 requested a full tablet report, sending...
I20260809 07:55:07.047531 7925 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:07.048013 8212 ts_manager.cc:194] Registered new tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:07.049281 8483 raft_consensus.cc:493] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a49f61c729f4bf5babc869212715595)
I20260809 07:55:07.049644 8483 raft_consensus.cc:515] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
I20260809 07:55:07.053814 7453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "79a0d69c482349298d8330b6de948fa4" is_pre_election: true
I20260809 07:55:07.054026 7453 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb74ca8d446c458599fc49b81a6c3bcd in term 1.
I20260809 07:55:07.054529 8479 leader_election.cc:290] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a49f61c729f4bf5babc869212715595 (127.26.37.62:34629), 79a0d69c482349298d8330b6de948fa4 (127.26.37.60:41925)
I20260809 07:55:07.054651 7326 leader_election.cc:304] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a0d69c482349298d8330b6de948fa4, bb74ca8d446c458599fc49b81a6c3bcd; no voters:
I20260809 07:55:07.055711 8463 heartbeater.cc:344] Connected to a master server at 127.26.37.60:41925
I20260809 07:55:07.055904 8463 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:07.056537 8463 heartbeater.cc:507] Master 127.26.37.60:41925 requested a full tablet report, sending...
I20260809 07:55:07.056811 8479 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:55:07.057915 7442 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:07.058159 8479 raft_consensus.cc:493] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a49f61c729f4bf5babc869212715595)
I20260809 07:55:07.058406 8479 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:07.059562 8465 heartbeater.cc:344] Connected to a master server at 127.26.37.61:33247
I20260809 07:55:07.059724 8465 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:07.060259 8465 heartbeater.cc:507] Master 127.26.37.61:33247 requested a full tablet report, sending...
I20260809 07:55:07.060912 8483 leader_election.cc:290] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a49f61c729f4bf5babc869212715595 (127.26.37.62:34629), bb74ca8d446c458599fc49b81a6c3bcd (127.26.37.61:33247)
I20260809 07:55:07.061710 7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79a0d69c482349298d8330b6de948fa4" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" is_pre_election: true
I20260809 07:55:07.065508 8479 raft_consensus.cc:515] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
I20260809 07:55:07.068331 8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5a49f61c729f4bf5babc869212715595" is_pre_election: true
I20260809 07:55:07.068810 7342 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:07.068929 8225 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb74ca8d446c458599fc49b81a6c3bcd in term 1.
I20260809 07:55:07.069474 7357 raft_consensus.cc:2393] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79a0d69c482349298d8330b6de948fa4 in current term 2: Already voted for candidate bb74ca8d446c458599fc49b81a6c3bcd in this term.
I20260809 07:55:07.070888 8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5a49f61c729f4bf5babc869212715595"
I20260809 07:55:07.071089 8225 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:07.071125 7453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "79a0d69c482349298d8330b6de948fa4"
I20260809 07:55:07.071331 7453 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:07.074736 7453 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb74ca8d446c458599fc49b81a6c3bcd in term 2.
I20260809 07:55:07.074718 8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79a0d69c482349298d8330b6de948fa4" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "5a49f61c729f4bf5babc869212715595" is_pre_election: true
I20260809 07:55:07.075253 7326 leader_election.cc:304] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a0d69c482349298d8330b6de948fa4, bb74ca8d446c458599fc49b81a6c3bcd; no voters:
I20260809 07:55:07.076567 8225 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb74ca8d446c458599fc49b81a6c3bcd in term 2.
I20260809 07:55:07.077440 8479 leader_election.cc:290] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [CANDIDATE]: Term 2 election: Requested vote from peers 5a49f61c729f4bf5babc869212715595 (127.26.37.62:34629), 79a0d69c482349298d8330b6de948fa4 (127.26.37.60:41925)
I20260809 07:55:07.077610 8485 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:55:07.077863 8485 raft_consensus.cc:697] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 2 LEADER]: Becoming Leader. State: Replica: bb74ca8d446c458599fc49b81a6c3bcd, State: Running, Role: LEADER
I20260809 07:55:07.078534 8485 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } }
I20260809 07:55:07.083581 7420 leader_election.cc:304] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79a0d69c482349298d8330b6de948fa4; no voters: 5a49f61c729f4bf5babc869212715595, bb74ca8d446c458599fc49b81a6c3bcd
I20260809 07:55:07.091287 8483 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:55:07.094324 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 8296
I20260809 07:55:07.109197 8479 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb74ca8d446c458599fc49b81a6c3bcd. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.109505 8479 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:07.109892 8455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:07.110272 8491 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:55:07.113261 8491 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:55:07.114492 8491 catalog_manager.cc:1295] Loaded cluster ID: f8a0cd79683b4219a6293b7edca6ee76
I20260809 07:55:07.114589 8491 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:55:07.115855 8491 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:55:07.116756 8491 catalog_manager.cc:6103] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: Loaded TSK: 0
I20260809 07:55:07.117813 8491 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:55:07.198062 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 8296
I20260809 07:55:07.223495 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35181
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--webserver_port=33659
--webserver_interface=127.26.37.1
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055
--fault_before_start=1.0 with env {}
I20260809 07:55:07.518149 7937 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 1 LEARNER]: Advancing to term 2
I20260809 07:55:07.522420 7937 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 2 LEARNER]: Refusing update from remote peer bb74ca8d446c458599fc49b81a6c3bcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 07:55:07.523512 8479 consensus_queue.cc:1048] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:55:07.527593 8494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb74ca8d446c458599fc49b81a6c3bcd. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.528151 8494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:07.528299 8479 raft_consensus.cc:1064] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: attempting to promote NON_VOTER 2a009e76a84f45a6b47befbc25cbbd84 to VOTER
W20260809 07:55:07.528627 8479 raft_consensus.cc:1068] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: Unable to promote non-voter 2a009e76a84f45a6b47befbc25cbbd84: Illegal state: Leader has not yet committed an operation in its own term
W20260809 07:55:07.587378 8493 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:07.587759 8493 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:07.587910 8493 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
I20260809 07:55:07.587921 7453 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 2 FOLLOWER]: Refusing update from remote peer bb74ca8d446c458599fc49b81a6c3bcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
W20260809 07:55:07.588006 8493 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:55:07.588665 8479 consensus_queue.cc:1048] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:55:07.590516 8483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb74ca8d446c458599fc49b81a6c3bcd. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.590765 8483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:07.593825 8225 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 2 FOLLOWER]: Refusing update from remote peer bb74ca8d446c458599fc49b81a6c3bcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 07:55:07.594825 8479 consensus_queue.cc:1048] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 07:55:07.603626 8493 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:07.603979 8493 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:07.608671 8494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.608917 8494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:07.614778 8485 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.615067 8485 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:07.615370 8479 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.615370 8483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.615626 8479 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:07.615638 8483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:07.615957 8485 raft_consensus.cc:1064] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd: attempting to promote NON_VOTER 2a009e76a84f45a6b47befbc25cbbd84 to VOTER
I20260809 07:55:07.617417 8485 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } }
I20260809 07:55:07.617509 8508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.617769 8508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:07.618487 8504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb74ca8d446c458599fc49b81a6c3bcd. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: true } } }
I20260809 07:55:07.618755 8504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:07.620252 7937 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 2 LEARNER]: Refusing update from remote peer bb74ca8d446c458599fc49b81a6c3bcd: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:07.620252 7453 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 2 FOLLOWER]: Refusing update from remote peer bb74ca8d446c458599fc49b81a6c3bcd: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:07.620886 8479 consensus_queue.cc:1048] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:07.621311 8225 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 2 FOLLOWER]: Refusing update from remote peer bb74ca8d446c458599fc49b81a6c3bcd: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:07.622061 8485 consensus_queue.cc:1048] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:07.622375 8509 consensus_queue.cc:1048] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:07.623790 8493 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35181
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33659
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:07.626106 8493 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:07.626353 8225 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 2a009e76a84f45a6b47befbc25cbbd84 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.626353 7937 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 2a009e76a84f45a6b47befbc25cbbd84 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.628588 8493 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:55:07.629730 8504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.629930 8504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a49f61c729f4bf5babc869212715595 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:07.629909 8501 raft_consensus.cc:2955] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 2a009e76a84f45a6b47befbc25cbbd84 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.630900 8494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.631088 8494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a009e76a84f45a6b47befbc25cbbd84 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:07.633536 8501 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.633641 8479 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.633764 8501 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:07.633800 8479 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:07.634007 8501 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.634225 8501 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:07.633977 7453 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 2a009e76a84f45a6b47befbc25cbbd84 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.634605 8485 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.634776 8485 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb74ca8d446c458599fc49b81a6c3bcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:07.638278 8483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5a49f61c729f4bf5babc869212715595" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34629 } } peers { permanent_uuid: "bb74ca8d446c458599fc49b81a6c3bcd" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 33247 } } peers { permanent_uuid: "79a0d69c482349298d8330b6de948fa4" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41925 } } peers { permanent_uuid: "2a009e76a84f45a6b47befbc25cbbd84" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43055 } attrs { promote: false } } }
I20260809 07:55:07.638569 8483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0d69c482349298d8330b6de948fa4 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:55:07.639634 8514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:07.640575 8512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:07.641618 8517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:07.642051 8493 server_base.cc:1061] running on GCE node
I20260809 07:55:07.642719 8493 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:07.642895 8493 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:07.643013 8493 hybrid_clock.cc:648] HybridClock initialized: now 1786262107642973 us; error 0 us; skew 500 ppm
I20260809 07:55:07.643321 8493 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:07.645537 8493 webserver.cc:533] Webserver started at http://127.26.37.1:33659/ using document root <none> and password file <none>
I20260809 07:55:07.646334 8493 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:07.646499 8493 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:07.651798 8493 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:55:07.654521 8524 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:07.655820 8493 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:55:07.655997 8493 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "94c054e210b84f2da1dc085da7055cbc"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:07.656747 8493 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:07.676294 8493 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:07.677284 8493 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:07.677615 8493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:07.678655 8493 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:07.680475 8493 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:07.680610 8493 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:07.680733 8493 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:07.680838 8493 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:55:07.681236 8493 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:55:07.681360 8493 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:55:07.681452 8493 fault_injection.cc:59] injecting fault for kudu (pid 8493): FLAGS_fault_before_start (process will exit)
I20260809 07:55:07.700479 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35181
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925
--webserver_port=33659
--webserver_interface=127.26.37.1
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055 with env {}
W20260809 07:55:08.059418 8536 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:08.059774 8536 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:08.059913 8536 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:08.068331 8536 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:08.068552 8536 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:08.078476 8536 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35181
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33659
--tserver_master_addrs=127.26.37.62:34629,127.26.37.61:33247,127.26.37.60:41925,127.26.37.59:43055
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:08.080255 8536 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:08.082273 8536 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:08.092120 8541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:08.092465 8542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:08.092872 8547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:08.093310 8536 server_base.cc:1061] running on GCE node
I20260809 07:55:08.093905 8536 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:08.094089 8536 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:08.094192 8536 hybrid_clock.cc:648] HybridClock initialized: now 1786262108094155 us; error 0 us; skew 500 ppm
I20260809 07:55:08.094504 8536 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:08.096611 8536 webserver.cc:533] Webserver started at http://127.26.37.1:33659/ using document root <none> and password file <none>
I20260809 07:55:08.097456 8536 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:08.097585 8536 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:08.103046 8536 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 07:55:08.105710 8554 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:08.107045 8536 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:55:08.107221 8536 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "94c054e210b84f2da1dc085da7055cbc"
format_stamp: "Formatted at 2026-08-09 07:55:04 on dist-test-slave-bw88"
I20260809 07:55:08.107918 8536 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:08.128727 8536 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:08.129698 8536 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:08.129994 8536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:08.130930 8536 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:08.132687 8536 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:08.132793 8536 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:08.132944 8536 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:08.133015 8536 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:08.170265 8536 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35181
I20260809 07:55:08.170315 8696 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35181 every 8 connection(s)
I20260809 07:55:08.172030 8536 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:08.176645 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 8536
I20260809 07:55:08.176946 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 8536
I20260809 07:55:08.201323 8700 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34629
I20260809 07:55:08.201823 8700 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:08.203305 8700 heartbeater.cc:507] Master 127.26.37.62:34629 requested a full tablet report, sending...
I20260809 07:55:08.206005 8212 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:08.212157 8701 heartbeater.cc:344] Connected to a master server at 127.26.37.61:33247
I20260809 07:55:08.212365 8701 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:08.213626 8701 heartbeater.cc:507] Master 127.26.37.61:33247 requested a full tablet report, sending...
I20260809 07:55:08.214689 7342 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:08.216564 7342 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:52155
I20260809 07:55:08.218500 8699 heartbeater.cc:344] Connected to a master server at 127.26.37.60:41925
I20260809 07:55:08.218638 8699 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:08.218937 8699 heartbeater.cc:507] Master 127.26.37.60:41925 requested a full tablet report, sending...
I20260809 07:55:08.219520 8697 heartbeater.cc:344] Connected to a master server at 127.26.37.59:43055
I20260809 07:55:08.219719 8697 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:08.220141 8697 heartbeater.cc:507] Master 127.26.37.59:43055 requested a full tablet report, sending...
I20260809 07:55:08.220219 7442 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:08.221566 7925 ts_manager.cc:194] Re-registered known tserver with Master: 94c054e210b84f2da1dc085da7055cbc (127.26.37.1:35181)
I20260809 07:55:08.226835 8691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:08.314697 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 8536
I20260809 07:55:08.339879 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 8176
I20260809 07:55:08.346626 8254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:08.450426 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 8176
W20260809 07:55:08.477913 7420 connection.cc:573] client connection to 127.26.37.62:34629 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:55:08.478273 7399 master_runner.cc:187] Error getting master registration for 127.26.37.62:34629: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260809 07:55:08.478484 7399 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:55:08.478382 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7304
I20260809 07:55:08.479723 7387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:08.592080 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7304
I20260809 07:55:08.620978 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7399
I20260809 07:55:08.622229 7481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:08.729328 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7399
I20260809 07:55:08.759336 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 7735
I20260809 07:55:08.760574 7963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:08.869029 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 7735
Shutting down hive metastore.
[ OK ] ExternalMiniClusterTest.TestAddMaster/3 (23405 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/4
2026-08-09T07:55:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:55:09Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:55:16: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:55:25.644357 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36555
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395 with env {}
W20260809 07:55:26.010138 9076 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:26.010622 9076 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:26.010762 9076 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:26.023916 9076 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:26.024040 9076 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:26.024137 9076 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:26.024263 9076 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:26.040473 9076 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:26.042817 9076 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:26.045051 9076 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:26.055716 9083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:26.055689 9084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:26.056519 9086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:26.065088 9076 server_base.cc:1061] running on GCE node
I20260809 07:55:26.066038 9076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:26.067588 9076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:26.068821 9076 hybrid_clock.cc:648] HybridClock initialized: now 1786262126068767 us; error 55 us; skew 500 ppm
I20260809 07:55:26.069206 9076 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:26.071514 9076 webserver.cc:533] Webserver started at http://127.26.37.62:46435/ using document root <none> and password file <none>
I20260809 07:55:26.072314 9076 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:26.072407 9076 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:26.072726 9076 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:26.075229 9076 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "cc4c5a10a6b842bf8696928b12c8848c"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:26.075989 9076 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "cc4c5a10a6b842bf8696928b12c8848c"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:26.081169 9076 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 07:55:26.084719 9094 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:26.086165 9076 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:55:26.086354 9076 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cc4c5a10a6b842bf8696928b12c8848c"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:26.086558 9076 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:26.113247 9076 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:26.114171 9076 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:26.114460 9076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:26.135926 9076 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36555
I20260809 07:55:26.136026 9165 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36555 every 8 connection(s)
I20260809 07:55:26.137573 9076 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:55:26.141861 9166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:26.143251 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9076
I20260809 07:55:26.143466 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:55:26.144825 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:41601
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395 with env {}
I20260809 07:55:26.150405 9166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:26.168016 9166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:26.169431 9098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:41601: connect: Connection refused (error 111)
W20260809 07:55:26.171295 9166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:41601: Network error: Client connection negotiation failed: client connection to 127.26.37.61:41601: connect: Connection refused (error 111)
I20260809 07:55:26.220679 9166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } attempt: 1
W20260809 07:55:26.222975 9166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:41601: Network error: Client connection negotiation failed: client connection to 127.26.37.61:41601: connect: Connection refused (error 111)
I20260809 07:55:26.291298 9166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } attempt: 2
W20260809 07:55:26.293644 9166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:41601: Network error: Client connection negotiation failed: client connection to 127.26.37.61:41601: connect: Connection refused (error 111)
I20260809 07:55:26.384912 9166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } attempt: 3
W20260809 07:55:26.387292 9166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:41601: Network error: Client connection negotiation failed: client connection to 127.26.37.61:41601: connect: Connection refused (error 111)
W20260809 07:55:26.496698 9171 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:26.497057 9171 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:26.497206 9171 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:26.505280 9171 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:26.505421 9171 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:26.505537 9171 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:26.505626 9171 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:26.515161 9171 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:41601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:26.516808 9171 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:26.518685 9171 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:26.526924 9181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:26.527096 9185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:26.526964 9182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:26.528736 9171 server_base.cc:1061] running on GCE node
I20260809 07:55:26.529410 9171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:26.530561 9166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } attempt: 4
I20260809 07:55:26.530690 9171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:26.532186 9171 hybrid_clock.cc:648] HybridClock initialized: now 1786262126532142 us; error 37 us; skew 500 ppm
W20260809 07:55:26.532469 9166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:41601: Network error: Client connection negotiation failed: client connection to 127.26.37.61:41601: connect: Connection refused (error 111)
I20260809 07:55:26.532640 9171 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:26.534770 9171 webserver.cc:533] Webserver started at http://127.26.37.61:37291/ using document root <none> and password file <none>
I20260809 07:55:26.535485 9171 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:26.535624 9171 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:26.536001 9171 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:26.538406 9171 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "dafecff6ae1d4269848f56e43cd57594"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:26.539230 9171 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "dafecff6ae1d4269848f56e43cd57594"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:26.544516 9171 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:55:26.547871 9193 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:26.549256 9171 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:55:26.549463 9171 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "dafecff6ae1d4269848f56e43cd57594"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:26.549685 9171 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:26.564616 9171 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:26.565471 9171 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:26.565747 9171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:26.586248 9171 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:41601
I20260809 07:55:26.586292 9253 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:41601 every 8 connection(s)
I20260809 07:55:26.587816 9171 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:55:26.592286 9254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:26.593470 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9171
I20260809 07:55:26.593688 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:55:26.594712 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:38395
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395 with env {}
I20260809 07:55:26.598692 9254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:26.613188 9254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:26.622932 9254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:26.624063 9196 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
W20260809 07:55:26.625746 9254 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:38395: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
I20260809 07:55:26.675153 9254 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } attempt: 1
W20260809 07:55:26.677415 9254 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:38395: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
I20260809 07:55:26.745749 9254 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } attempt: 2
W20260809 07:55:26.747867 9254 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:38395: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
I20260809 07:55:26.831718 9166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } attempt: 5
I20260809 07:55:26.838738 9166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:26.839089 9254 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } attempt: 3
W20260809 07:55:26.841007 9166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:38395: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
W20260809 07:55:26.841436 9254 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:38395: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
I20260809 07:55:26.892236 9166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } attempt: 1
W20260809 07:55:26.894479 9166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:38395: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
W20260809 07:55:26.955787 9258 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:26.956130 9258 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:26.956203 9258 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260809 07:55:26.962716 9166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } attempt: 2
W20260809 07:55:26.964691 9258 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:26.964838 9258 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:26.964881 9166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:38395: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
W20260809 07:55:26.964963 9258 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:26.965029 9258 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:26.974699 9258 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:38395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:26.976328 9258 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:26.978296 9258 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:55:26.984673 9254 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } attempt: 4
W20260809 07:55:26.987293 9254 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:38395: Network error: Client connection negotiation failed: client connection to 127.26.37.60:38395: connect: Connection refused (error 111)
W20260809 07:55:26.989650 9270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:26.989729 9268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:26.989751 9265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:26.991715 9258 server_base.cc:1061] running on GCE node
I20260809 07:55:26.992517 9258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:26.993960 9258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:26.995285 9258 hybrid_clock.cc:648] HybridClock initialized: now 1786262126995219 us; error 44 us; skew 500 ppm
I20260809 07:55:26.995652 9258 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:26.997977 9258 webserver.cc:533] Webserver started at http://127.26.37.60:40821/ using document root <none> and password file <none>
I20260809 07:55:26.998699 9258 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:26.998833 9258 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:26.999225 9258 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:27.001699 9258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "0e58bc7a7bc147319d4b5db97d7935f0"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:27.002467 9258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "0e58bc7a7bc147319d4b5db97d7935f0"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:27.007757 9258 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:55:27.010982 9277 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:27.012270 9258 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:55:27.012457 9258 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "0e58bc7a7bc147319d4b5db97d7935f0"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:27.012673 9258 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:27.029773 9258 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:27.030527 9258 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:27.030795 9258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:27.051384 9258 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:38395
I20260809 07:55:27.051403 9343 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:38395 every 8 connection(s)
I20260809 07:55:27.052983 9258 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:55:27.056555 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9258
I20260809 07:55:27.056754 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:55:27.057621 9344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:27.065491 9344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:27.071135 9166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } attempt: 3
I20260809 07:55:27.080199 9344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:27.085319 9166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Bootstrap starting.
I20260809 07:55:27.089097 9344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:27.090133 9166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:27.091811 9166 log.cc:826] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:27.095876 9166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: No bootstrap required, opened a new log
I20260809 07:55:27.101289 9344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: Bootstrap starting.
I20260809 07:55:27.101975 9166 raft_consensus.cc:359] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
I20260809 07:55:27.102234 9166 raft_consensus.cc:385] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:27.102391 9166 raft_consensus.cc:740] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4c5a10a6b842bf8696928b12c8848c, State: Initialized, Role: FOLLOWER
I20260809 07:55:27.102958 9166 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
I20260809 07:55:27.104059 9352 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.104348 9352 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:27.104812 9344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:27.105263 9166 sys_catalog.cc:565] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:27.105998 9344 log.cc:826] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:27.108989 9344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: No bootstrap required, opened a new log
I20260809 07:55:27.115113 9344 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
I20260809 07:55:27.115448 9344 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:27.115617 9344 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e58bc7a7bc147319d4b5db97d7935f0, State: Initialized, Role: FOLLOWER
I20260809 07:55:27.116344 9344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
W20260809 07:55:27.117475 9101 tablet.cc:2406] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:55:27.119115 9365 catalog_manager.cc:1594] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:27.119256 9365 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:27.120601 9367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.120942 9367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:27.121852 9344 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:27.121206 9076 master_runner.cc:187] Error getting master registration for 127.26.37.61:41601: OK, instance_id { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" instance_seqno: 1786262126580685 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:55:27.127802 9076 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:55:27.133911 9379 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:27.134189 9379 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:27.141994 9258 master_runner.cc:187] Error getting master registration for 127.26.37.61:41601: OK, instance_id { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" instance_seqno: 1786262126580685 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:55:27.148365 9258 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:55:27.282799 9284 tablet.cc:2406] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:55:27.286540 9254 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } attempt: 5
I20260809 07:55:27.299409 9254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: Bootstrap starting.
I20260809 07:55:27.303722 9254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:27.305052 9254 log.cc:826] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:27.308364 9254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: No bootstrap required, opened a new log
I20260809 07:55:27.315878 9254 raft_consensus.cc:359] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
I20260809 07:55:27.316299 9254 raft_consensus.cc:385] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:27.316417 9254 raft_consensus.cc:740] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafecff6ae1d4269848f56e43cd57594, State: Initialized, Role: FOLLOWER
I20260809 07:55:27.318434 9254 consensus_queue.cc:260] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
I20260809 07:55:27.319936 9382 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.320286 9382 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:27.321179 9254 sys_catalog.cc:565] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:27.335561 9171 master_runner.cc:264] No leader master found from master dafecff6ae1d4269848f56e43cd57594
I20260809 07:55:27.335676 9171 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:55:27.338428 9395 catalog_manager.cc:1594] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:27.338625 9395 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:27.343286 9352 raft_consensus.cc:493] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:55:27.343567 9352 raft_consensus.cc:515] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
I20260809 07:55:27.345306 9352 leader_election.cc:290] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dafecff6ae1d4269848f56e43cd57594 (127.26.37.61:41601), 0e58bc7a7bc147319d4b5db97d7935f0 (127.26.37.60:38395)
I20260809 07:55:27.346249 9308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc4c5a10a6b842bf8696928b12c8848c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" is_pre_election: true
I20260809 07:55:27.346228 9224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc4c5a10a6b842bf8696928b12c8848c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafecff6ae1d4269848f56e43cd57594" is_pre_election: true
I20260809 07:55:27.346696 9224 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4c5a10a6b842bf8696928b12c8848c in term 0.
I20260809 07:55:27.346706 9308 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4c5a10a6b842bf8696928b12c8848c in term 0.
I20260809 07:55:27.347358 9098 leader_election.cc:304] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4c5a10a6b842bf8696928b12c8848c, dafecff6ae1d4269848f56e43cd57594; no voters:
I20260809 07:55:27.348219 9352 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:55:27.348372 9352 raft_consensus.cc:493] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:55:27.348469 9352 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:55:27.351680 9352 raft_consensus.cc:515] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
I20260809 07:55:27.352720 9224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc4c5a10a6b842bf8696928b12c8848c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafecff6ae1d4269848f56e43cd57594"
I20260809 07:55:27.352911 9224 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:55:27.356312 9224 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4c5a10a6b842bf8696928b12c8848c in term 1.
I20260809 07:55:27.356817 9098 leader_election.cc:304] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4c5a10a6b842bf8696928b12c8848c, dafecff6ae1d4269848f56e43cd57594; no voters:
I20260809 07:55:27.360682 9308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc4c5a10a6b842bf8696928b12c8848c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e58bc7a7bc147319d4b5db97d7935f0"
I20260809 07:55:27.360885 9308 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:55:27.364367 9308 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4c5a10a6b842bf8696928b12c8848c in term 1.
I20260809 07:55:27.365067 9352 leader_election.cc:290] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [CANDIDATE]: Term 1 election: Requested vote from peers dafecff6ae1d4269848f56e43cd57594 (127.26.37.61:41601), 0e58bc7a7bc147319d4b5db97d7935f0 (127.26.37.60:38395)
I20260809 07:55:27.365274 9352 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:55:27.365424 9352 raft_consensus.cc:697] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 LEADER]: Becoming Leader. State: Replica: cc4c5a10a6b842bf8696928b12c8848c, State: Running, Role: LEADER
I20260809 07:55:27.365923 9352 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } }
I20260809 07:55:27.381659 9352 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc4c5a10a6b842bf8696928b12c8848c. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.381943 9352 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:27.385228 9399 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:55:27.387787 9399 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:55:27.406271 9308 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Refusing update from remote peer cc4c5a10a6b842bf8696928b12c8848c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:55:27.407418 9397 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:55:27.418859 9224 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Refusing update from remote peer cc4c5a10a6b842bf8696928b12c8848c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:55:27.419866 9397 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:55:27.452409 9382 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc4c5a10a6b842bf8696928b12c8848c. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.452723 9382 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:27.456348 9400 mvcc.cc:204] Tried to move back new op lower bound from 7316529673841758208 to 7316529673713430528. Current Snapshot: MvccSnapshot[applied={T|T < 7316529673841758208}]
I20260809 07:55:27.462069 9402 mvcc.cc:204] Tried to move back new op lower bound from 7316529673841758208 to 7316529673713430528. Current Snapshot: MvccSnapshot[applied={T|T < 7316529673841758208}]
I20260809 07:55:27.467900 9352 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.468189 9352 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:27.471026 9399 catalog_manager.cc:1383] Generated new cluster ID: 870aa0d0a4404e7f8f6691207a556320
I20260809 07:55:27.471138 9399 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:55:27.474095 9382 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.474319 9382 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:27.488600 9367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc4c5a10a6b842bf8696928b12c8848c. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.488869 9367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:27.494025 9397 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.494365 9397 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:27.496291 9401 mvcc.cc:204] Tried to move back new op lower bound from 7316529673841758208 to 7316529673713430528. Current Snapshot: MvccSnapshot[applied={T|T < 7316529673841758208}]
I20260809 07:55:27.510811 9367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } }
I20260809 07:55:27.511023 9367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:27.537371 9399 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:55:27.543541 9399 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:55:27.562268 9399 catalog_manager.cc:6092] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Generated new TSK 0
I20260809 07:55:27.563550 9399 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:55:27.577332 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:55:27.959885 9409 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:27.960161 9409 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:27.960278 9409 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:27.968060 9409 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:27.968215 9409 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:27.977691 9409 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:27.979206 9409 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:27.981096 9409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:27.991380 9416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:27.991329 9415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:27.991530 9418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:27.992760 9409 server_base.cc:1061] running on GCE node
I20260809 07:55:27.993520 9409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:27.994467 9409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:27.995718 9409 hybrid_clock.cc:648] HybridClock initialized: now 1786262127995654 us; error 55 us; skew 500 ppm
I20260809 07:55:27.996050 9409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:27.998350 9409 webserver.cc:533] Webserver started at http://127.26.37.1:42957/ using document root <none> and password file <none>
I20260809 07:55:27.999115 9409 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:27.999261 9409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:27.999658 9409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:28.002315 9409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "157a5987150c4fdcbcf0dc9bb25bd25c"
format_stamp: "Formatted at 2026-08-09 07:55:27 on dist-test-slave-bw88"
I20260809 07:55:28.003171 9409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "157a5987150c4fdcbcf0dc9bb25bd25c"
format_stamp: "Formatted at 2026-08-09 07:55:27 on dist-test-slave-bw88"
I20260809 07:55:28.009048 9409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:55:28.012705 9426 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:28.014068 9409 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:55:28.014258 9409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "157a5987150c4fdcbcf0dc9bb25bd25c"
format_stamp: "Formatted at 2026-08-09 07:55:27 on dist-test-slave-bw88"
I20260809 07:55:28.014509 9409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:28.029541 9409 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:28.030385 9409 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:28.030691 9409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:28.031675 9409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:28.033232 9409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:28.033367 9409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:28.033501 9409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:28.033596 9409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:28.069660 9409 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34475
I20260809 07:55:28.069702 9563 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34475 every 8 connection(s)
I20260809 07:55:28.071171 9409 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:28.072255 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9409
I20260809 07:55:28.072441 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:55:28.100383 9566 heartbeater.cc:344] Connected to a master server at 127.26.37.61:41601
I20260809 07:55:28.100812 9566 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:28.101029 9564 heartbeater.cc:344] Connected to a master server at 127.26.37.60:38395
I20260809 07:55:28.101181 9564 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:28.102012 9565 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36555
I20260809 07:55:28.102159 9565 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:28.102123 9564 heartbeater.cc:507] Master 127.26.37.60:38395 requested a full tablet report, sending...
I20260809 07:55:28.102733 9565 heartbeater.cc:507] Master 127.26.37.62:36555 requested a full tablet report, sending...
I20260809 07:55:28.102154 9566 heartbeater.cc:507] Master 127.26.37.61:41601 requested a full tablet report, sending...
I20260809 07:55:28.104578 9115 ts_manager.cc:194] Registered new tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:28.104856 9211 ts_manager.cc:194] Registered new tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:28.104853 9297 ts_manager.cc:194] Registered new tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:28.106537 9115 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:53605
I20260809 07:55:28.119004 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:28.121332 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:38257
--webserver_interface=127.26.37.59
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257 with env {}
I20260809 07:55:28.136127 9379 catalog_manager.cc:1295] Loaded cluster ID: 870aa0d0a4404e7f8f6691207a556320
I20260809 07:55:28.136242 9379 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: loading cluster ID for follower catalog manager: success
I20260809 07:55:28.139611 9379 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: acquiring CA information for follower catalog manager: success
I20260809 07:55:28.142148 9379 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:55:28.340593 9395 catalog_manager.cc:1295] Loaded cluster ID: 870aa0d0a4404e7f8f6691207a556320
I20260809 07:55:28.340692 9395 catalog_manager.cc:1588] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: loading cluster ID for follower catalog manager: success
I20260809 07:55:28.343453 9395 catalog_manager.cc:1610] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: acquiring CA information for follower catalog manager: success
I20260809 07:55:28.345315 9395 catalog_manager.cc:1638] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:55:28.473920 9575 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:28.474318 9575 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:28.474458 9575 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:28.482540 9575 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:28.482651 9575 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:28.482774 9575 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:28.482856 9575 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:28.492395 9575 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:38257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:28.494058 9575 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:28.495844 9575 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:28.503629 9582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:28.503719 9581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:28.504671 9586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:28.505154 9575 server_base.cc:1061] running on GCE node
I20260809 07:55:28.505889 9575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:28.507550 9575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:28.508740 9575 hybrid_clock.cc:648] HybridClock initialized: now 1786262128508709 us; error 34 us; skew 500 ppm
I20260809 07:55:28.509064 9575 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:28.511130 9575 webserver.cc:533] Webserver started at http://127.26.37.59:37955/ using document root <none> and password file <none>
I20260809 07:55:28.511751 9575 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:28.511837 9575 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:28.512283 9575 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:28.514626 9575 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9"
format_stamp: "Formatted at 2026-08-09 07:55:28 on dist-test-slave-bw88"
I20260809 07:55:28.515314 9575 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9"
format_stamp: "Formatted at 2026-08-09 07:55:28 on dist-test-slave-bw88"
I20260809 07:55:28.520460 9575 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:55:28.523768 9593 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:28.525106 9575 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:55:28.525271 9575 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9"
format_stamp: "Formatted at 2026-08-09 07:55:28 on dist-test-slave-bw88"
I20260809 07:55:28.525542 9575 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:28.553571 9575 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:28.554387 9575 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:28.554669 9575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:28.576794 9575 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:38257
I20260809 07:55:28.576815 9655 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:38257 every 8 connection(s)
I20260809 07:55:28.578851 9575 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:55:28.580546 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9575
I20260809 07:55:28.584162 9656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:28.594892 9656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:28.609378 9656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:28.616577 9656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:28.623049 9656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:28.633904 9656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Bootstrap starting.
I20260809 07:55:28.637107 9656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:28.638259 9656 log.cc:826] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:28.640959 9656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: No bootstrap required, opened a new log
I20260809 07:55:28.646091 9656 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } }
I20260809 07:55:28.646433 9656 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:28.646562 9656 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0fc88f56a44864a2cbcd4613e4d6a9, State: Initialized, Role: FOLLOWER
I20260809 07:55:28.647260 9656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } }
I20260809 07:55:28.648384 9665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } } }
I20260809 07:55:28.648869 9665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:28.649623 9656 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:55:28.660923 9678 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:28.661113 9678 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:28.666915 9575 master_runner.cc:429] Detected that this master 0b0fc88f56a44864a2cbcd4613e4d6a9 is joining an existing cluster
I20260809 07:55:28.667027 9575 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:55:28.674263 9575 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:55:28.674599 9575 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:55:28.674811 9575 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: stopping tablet replica
I20260809 07:55:28.676460 9575 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:38257
I20260809 07:55:28.710601 9115 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:38257: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 38257
}
attrs {
promote: true
}
}
dest_uuid: "cc4c5a10a6b842bf8696928b12c8848c"
cas_config_opid_index: -1
I20260809 07:55:28.711985 9115 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } }
I20260809 07:55:28.714845 9223 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Refusing update from remote peer cc4c5a10a6b842bf8696928b12c8848c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260809 07:55:28.715453 9095 consensus_peers.cc:597] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c -> Peer 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59:38257): Couldn't send request to peer 0b0fc88f56a44864a2cbcd4613e4d6a9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:55:28.715616 9407 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:55:28.719645 9407 raft_consensus.cc:2955] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:28.720305 9223 raft_consensus.cc:2955] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:28.724283 9407 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:38257
I20260809 07:55:28.725219 9575 master.cc:562] Master@127.26.37.59:38257 shutting down...
I20260809 07:55:28.730335 9307 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Refusing update from remote peer cc4c5a10a6b842bf8696928b12c8848c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:55:28.731626 9694 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:55:28.742816 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:28.742731 9307 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:28.743777 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:28.743451 9691 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:28.744210 9691 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:28.755172 9700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:28.755467 9700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:28.756812 9698 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:28.757205 9698 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:55:28.769122 8729 connection.cc:573] client connection to 127.26.37.59:38257 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:55:28.782295 9575 master.cc:584] Master@127.26.37.59:38257 shutdown complete.
I20260809 07:55:28.782423 9575 master_runner.cc:306] Clearing existing system tablet
I20260809 07:55:28.782822 9575 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:28.782939 9575 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:28.785743 9575 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:55:28.787388 9702 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:28.787741 9575 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:28.788014 9575 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 07:55:28.788170 9575 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9"
format_stamp: "Formatted at 2026-08-09 07:55:28 on dist-test-slave-bw88"
I20260809 07:55:28.791436 9575 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:55:28.796278 9575 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:55:28.796442 9575 log.cc:1199] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:55:28.796885 9575 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Deleting consensus metadata
I20260809 07:55:28.797142 9575 master_runner.cc:316] Copying system tablet from 127.26.37.62:36555
I20260809 07:55:28.799714 9575 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.62:36555
I20260809 07:55:28.806650 9147 tablet_copy_service.cc:140] P cc4c5a10a6b842bf8696928b12c8848c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0b0fc88f56a44864a2cbcd4613e4d6a9 ({username='slave'} at 127.0.0.1:57056)
I20260809 07:55:28.806901 9147 tablet_copy_service.cc:161] P cc4c5a10a6b842bf8696928b12c8848c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0b0fc88f56a44864a2cbcd4613e4d6a9 at {username='slave'} at 127.0.0.1:57056: session id = 0b0fc88f56a44864a2cbcd4613e4d6a9-00000000000000000000000000000000
I20260809 07:55:28.809924 9147 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:55:28.811816 9575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:55:28.816377 9575 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: tablet copy: Starting download of 0 data blocks...
I20260809 07:55:28.816628 9575 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: tablet copy: Starting download of 1 WAL segments...
I20260809 07:55:28.819195 9575 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:55:28.822883 9147 tablet_copy_service.cc:342] P cc4c5a10a6b842bf8696928b12c8848c: Request end of tablet copy session 0b0fc88f56a44864a2cbcd4613e4d6a9-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:57056
I20260809 07:55:28.823104 9147 tablet_copy_service.cc:434] P cc4c5a10a6b842bf8696928b12c8848c: ending tablet copy session 0b0fc88f56a44864a2cbcd4613e4d6a9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0b0fc88f56a44864a2cbcd4613e4d6a9
W20260809 07:55:28.829893 9575 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260809 07:55:28.833118 9575 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:28.835752 9709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:28.836262 9710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:28.838380 9712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:28.838681 9575 server_base.cc:1061] running on GCE node
I20260809 07:55:28.839001 9575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:28.839594 9575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:28.840806 9575 hybrid_clock.cc:648] HybridClock initialized: now 1786262128840756 us; error 30 us; skew 500 ppm
I20260809 07:55:28.841003 9575 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:28.842569 9575 webserver.cc:533] Webserver started at http://127.26.37.59:42533/ using document root <none> and password file <none>
I20260809 07:55:28.842888 9575 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:28.842996 9575 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:28.845156 9575 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 07:55:28.846935 9719 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:28.847365 9575 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 07:55:28.847506 9575 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9"
format_stamp: "Formatted at 2026-08-09 07:55:28 on dist-test-slave-bw88"
I20260809 07:55:28.847667 9575 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:28.860601 9575 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:28.861294 9575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:28.883103 9575 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:38257
I20260809 07:55:28.883152 9787 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:38257 every 8 connection(s)
I20260809 07:55:28.884030 9575 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:55:28.888698 9788 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:55:28.892300 9788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Bootstrap starting.
I20260809 07:55:28.909183 9788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:55:28.909756 9788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: Bootstrap complete.
I20260809 07:55:28.910909 9788 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } }
I20260809 07:55:28.911181 9788 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0b0fc88f56a44864a2cbcd4613e4d6a9, State: Initialized, Role: LEARNER
I20260809 07:55:28.911473 9788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } }
I20260809 07:55:28.912334 9791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:28.912622 9791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:28.912679 9788 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:28.921638 9805 catalog_manager.cc:1295] Loaded cluster ID: 870aa0d0a4404e7f8f6691207a556320
I20260809 07:55:28.921716 9805 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: loading cluster ID for follower catalog manager: success
I20260809 07:55:28.925599 9805 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: acquiring CA information for follower catalog manager: success
I20260809 07:55:28.927703 9805 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:55:28.930253 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9409
I20260809 07:55:28.938822 9558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:29.022861 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9409
I20260809 07:55:29.049646 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34475
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--webserver_port=42957
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257 with env {}
I20260809 07:55:29.101063 9752 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:55:29.102250 9791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc4c5a10a6b842bf8696928b12c8848c. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:29.102499 9791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:29.345599 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:29.345975 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
W20260809 07:55:29.413518 9810 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:29.413801 9810 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:29.413932 9810 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:29.422158 9810 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:29.422302 9810 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:29.432204 9810 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34475
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42957
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:29.433799 9810 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:29.435701 9810 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:29.446117 9820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:29.446192 9819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:29.446700 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:29.447377 9810 server_base.cc:1061] running on GCE node
I20260809 07:55:29.448055 9810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:29.449165 9810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:29.450393 9810 hybrid_clock.cc:648] HybridClock initialized: now 1786262129450350 us; error 55 us; skew 500 ppm
I20260809 07:55:29.450734 9810 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:29.452838 9810 webserver.cc:533] Webserver started at http://127.26.37.1:42957/ using document root <none> and password file <none>
I20260809 07:55:29.453665 9810 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:29.453840 9810 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:29.459224 9810 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:55:29.461836 9834 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:29.463017 9810 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:29.463188 9810 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "157a5987150c4fdcbcf0dc9bb25bd25c"
format_stamp: "Formatted at 2026-08-09 07:55:27 on dist-test-slave-bw88"
I20260809 07:55:29.463877 9810 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:29.481853 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: true } } }
I20260809 07:55:29.482147 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:29.483538 9407 raft_consensus.cc:1064] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: attempting to promote NON_VOTER 0b0fc88f56a44864a2cbcd4613e4d6a9 to VOTER
I20260809 07:55:29.484484 9407 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } }
I20260809 07:55:29.485875 9810 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:29.486702 9307 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Refusing update from remote peer cc4c5a10a6b842bf8696928b12c8848c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:55:29.486886 9810 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:29.486876 9752 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 LEARNER]: Refusing update from remote peer cc4c5a10a6b842bf8696928b12c8848c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:55:29.486995 9223 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Refusing update from remote peer cc4c5a10a6b842bf8696928b12c8848c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:55:29.487212 9810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:29.487535 9811 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:55:29.487831 9812 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:55:29.488413 9810 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:29.488327 9812 consensus_queue.cc:1048] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:55:29.491025 9810 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:29.491144 9810 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:29.491295 9810 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:29.491374 9810 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:29.490947 9407 raft_consensus.cc:2955] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.491628 9223 raft_consensus.cc:2955] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.495513 9811 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.495781 9811 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:29.496729 9811 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.496950 9811 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:29.496941 9307 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.497349 9811 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.497560 9811 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:29.501890 9752 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.506047 9845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.506404 9845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:29.507845 9791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.508185 9791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:29.508406 9844 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4c5a10a6b842bf8696928b12c8848c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:29.508719 9844 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:29.544395 9810 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34475
I20260809 07:55:29.544472 9988 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34475 every 8 connection(s)
I20260809 07:55:29.545998 9810 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:29.550022 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 9810
I20260809 07:55:29.574342 9993 heartbeater.cc:344] Connected to a master server at 127.26.37.61:41601
I20260809 07:55:29.574788 9993 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:29.575548 9989 heartbeater.cc:344] Connected to a master server at 127.26.37.59:38257
I20260809 07:55:29.575737 9989 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:29.576115 9993 heartbeater.cc:507] Master 127.26.37.61:41601 requested a full tablet report, sending...
I20260809 07:55:29.576236 9989 heartbeater.cc:507] Master 127.26.37.59:38257 requested a full tablet report, sending...
I20260809 07:55:29.578773 9211 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:29.578969 9741 ts_manager.cc:194] Registered new tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:29.580669 9992 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36555
I20260809 07:55:29.580816 9992 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:29.581317 9990 heartbeater.cc:344] Connected to a master server at 127.26.37.60:38395
I20260809 07:55:29.581423 9992 heartbeater.cc:507] Master 127.26.37.62:36555 requested a full tablet report, sending...
I20260809 07:55:29.581547 9990 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:29.582079 9990 heartbeater.cc:507] Master 127.26.37.60:38395 requested a full tablet report, sending...
I20260809 07:55:29.582685 9115 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:29.583315 9297 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:29.583649 9115 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:49499
I20260809 07:55:29.591943 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:29.599713 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:29.599895 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9076
I20260809 07:55:29.600728 9160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:29.711939 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9076
I20260809 07:55:29.741743 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36555
--webserver_interface=127.26.37.62
--webserver_port=46435
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257 with env {}
W20260809 07:55:30.093400 10007 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:30.093726 10007 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:30.093863 10007 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:30.102056 10007 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:30.102137 10007 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:30.102241 10007 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:30.102320 10007 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:30.111989 10007 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=46435
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:30.113586 10007 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:30.115355 10007 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:30.123492 10013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:30.123468 10014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:30.124269 10016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:30.125314 10007 server_base.cc:1061] running on GCE node
I20260809 07:55:30.126158 10007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:30.127513 10007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:30.128762 10007 hybrid_clock.cc:648] HybridClock initialized: now 1786262130128720 us; error 53 us; skew 500 ppm
I20260809 07:55:30.129091 10007 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:30.131161 10007 webserver.cc:533] Webserver started at http://127.26.37.62:46435/ using document root <none> and password file <none>
I20260809 07:55:30.131847 10007 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:30.132005 10007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:30.137326 10007 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:30.140136 10026 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:30.141525 10007 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:55:30.141695 10007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cc4c5a10a6b842bf8696928b12c8848c"
format_stamp: "Formatted at 2026-08-09 07:55:26 on dist-test-slave-bw88"
I20260809 07:55:30.142467 10007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:30.159868 10007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:30.160684 10007 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:30.160988 10007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:30.181774 10007 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36555
I20260809 07:55:30.181809 10095 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36555 every 8 connection(s)
I20260809 07:55:30.183403 10007 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:55:30.190358 10097 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:55:30.190557 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 10007
I20260809 07:55:30.190932 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9810
I20260809 07:55:30.194401 10097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Bootstrap starting.
I20260809 07:55:30.199856 9984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:30.213660 10097 log.cc:826] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:30.223779 10097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:55:30.224311 10097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: Bootstrap complete.
I20260809 07:55:30.231014 10097 raft_consensus.cc:359] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } }
I20260809 07:55:30.231739 10097 raft_consensus.cc:740] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4c5a10a6b842bf8696928b12c8848c, State: Initialized, Role: FOLLOWER
I20260809 07:55:30.232435 10097 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } }
I20260809 07:55:30.233395 10104 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:30.233738 10104 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:30.234501 10097 sys_catalog.cc:565] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:30.245206 10119 catalog_manager.cc:1295] Loaded cluster ID: 870aa0d0a4404e7f8f6691207a556320
I20260809 07:55:30.245373 10119 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: loading cluster ID for follower catalog manager: success
I20260809 07:55:30.249666 10119 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: acquiring CA information for follower catalog manager: success
I20260809 07:55:30.252127 10119 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:55:30.297894 10007 master_runner.cc:264] No leader master found from master cc4c5a10a6b842bf8696928b12c8848c
I20260809 07:55:30.298087 10007 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:55:30.302152 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9810
I20260809 07:55:30.328068 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34475
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--webserver_port=42957
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257 with env {}
W20260809 07:55:30.699432 10128 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:30.699754 10128 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:30.699909 10128 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:30.708148 10128 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:30.708375 10128 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:30.718281 10128 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34475
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42957
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:30.719791 10128 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:30.721613 10128 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:30.731357 10135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:30.731379 10134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:30.732332 10138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:30.732395 10128 server_base.cc:1061] running on GCE node
I20260809 07:55:30.732963 10128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:30.733954 10128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:30.735189 10128 hybrid_clock.cc:648] HybridClock initialized: now 1786262130735141 us; error 43 us; skew 500 ppm
I20260809 07:55:30.735482 10128 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:30.737586 10128 webserver.cc:533] Webserver started at http://127.26.37.1:42957/ using document root <none> and password file <none>
I20260809 07:55:30.738354 10128 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:30.738500 10128 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:30.743688 10128 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 07:55:30.746421 10145 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:30.747758 10128 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:30.747921 10128 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "157a5987150c4fdcbcf0dc9bb25bd25c"
format_stamp: "Formatted at 2026-08-09 07:55:27 on dist-test-slave-bw88"
I20260809 07:55:30.748658 10128 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:30.771652 10128 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:30.772444 10128 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:30.772747 10128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:30.773690 10128 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:30.775527 10128 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:30.775635 10128 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:30.775786 10128 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:30.775873 10128 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:30.810827 10128 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34475
I20260809 07:55:30.810913 10283 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34475 every 8 connection(s)
I20260809 07:55:30.812376 10128 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:30.815663 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 10128
I20260809 07:55:30.861747 10287 heartbeater.cc:344] Connected to a master server at 127.26.37.60:38395
I20260809 07:55:30.862159 10287 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:30.863379 10287 heartbeater.cc:507] Master 127.26.37.60:38395 requested a full tablet report, sending...
I20260809 07:55:30.863624 10284 heartbeater.cc:344] Connected to a master server at 127.26.37.59:38257
I20260809 07:55:30.863811 10284 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:30.864315 10284 heartbeater.cc:507] Master 127.26.37.59:38257 requested a full tablet report, sending...
I20260809 07:55:30.865300 10289 heartbeater.cc:344] Connected to a master server at 127.26.37.61:41601
I20260809 07:55:30.865470 10289 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:30.865690 10288 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36555
I20260809 07:55:30.865722 9741 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:30.865864 10288 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:30.866035 10289 heartbeater.cc:507] Master 127.26.37.61:41601 requested a full tablet report, sending...
I20260809 07:55:30.866372 10288 heartbeater.cc:507] Master 127.26.37.62:36555 requested a full tablet report, sending...
I20260809 07:55:30.867157 9297 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:30.867437 9211 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:30.867852 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 10128
I20260809 07:55:30.869733 10051 ts_manager.cc:194] Registered new tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:30.880293 10277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:30.937438 10309 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc4c5a10a6b842bf8696928b12c8848c)
I20260809 07:55:30.937781 10309 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } }
I20260809 07:55:30.939661 10309 leader_election.cc:290] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc4c5a10a6b842bf8696928b12c8848c (127.26.37.62:36555), dafecff6ae1d4269848f56e43cd57594 (127.26.37.61:41601), 0e58bc7a7bc147319d4b5db97d7935f0 (127.26.37.60:38395)
I20260809 07:55:30.951943 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "cc4c5a10a6b842bf8696928b12c8848c" is_pre_election: true
I20260809 07:55:30.952567 10062 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b0fc88f56a44864a2cbcd4613e4d6a9 in term 1.
I20260809 07:55:30.958776 9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "dafecff6ae1d4269848f56e43cd57594" is_pre_election: true
I20260809 07:55:30.960336 9307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" is_pre_election: true
I20260809 07:55:30.961004 9723 leader_election.cc:304] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0b0fc88f56a44864a2cbcd4613e4d6a9, cc4c5a10a6b842bf8696928b12c8848c; no voters: 0e58bc7a7bc147319d4b5db97d7935f0, dafecff6ae1d4269848f56e43cd57594
I20260809 07:55:30.961489 10309 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:55:30.974846 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 10128
I20260809 07:55:30.993003 10315 raft_consensus.cc:493] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc4c5a10a6b842bf8696928b12c8848c)
I20260809 07:55:30.993243 10315 raft_consensus.cc:515] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } }
I20260809 07:55:30.995098 10315 leader_election.cc:290] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc4c5a10a6b842bf8696928b12c8848c (127.26.37.62:36555), 0e58bc7a7bc147319d4b5db97d7935f0 (127.26.37.60:38395), 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59:38257)
I20260809 07:55:30.996449 9307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafecff6ae1d4269848f56e43cd57594" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" is_pre_election: true
I20260809 07:55:31.000128 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34475
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--webserver_port=42957
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257
--fault_before_start=1.0 with env {}
I20260809 07:55:31.000134 10319 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc4c5a10a6b842bf8696928b12c8848c)
I20260809 07:55:31.000355 10319 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } }
I20260809 07:55:31.005934 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafecff6ae1d4269848f56e43cd57594" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "cc4c5a10a6b842bf8696928b12c8848c" is_pre_election: true
I20260809 07:55:31.006183 10062 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafecff6ae1d4269848f56e43cd57594 in term 1.
I20260809 07:55:31.006876 9752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafecff6ae1d4269848f56e43cd57594" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" is_pre_election: true
I20260809 07:55:31.007395 9752 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafecff6ae1d4269848f56e43cd57594 in term 1.
I20260809 07:55:31.007591 9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "dafecff6ae1d4269848f56e43cd57594" is_pre_election: true
I20260809 07:55:31.007836 9223 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e58bc7a7bc147319d4b5db97d7935f0 in term 1.
I20260809 07:55:31.008069 9194 leader_election.cc:304] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0b0fc88f56a44864a2cbcd4613e4d6a9, cc4c5a10a6b842bf8696928b12c8848c, dafecff6ae1d4269848f56e43cd57594; no voters: 0e58bc7a7bc147319d4b5db97d7935f0
I20260809 07:55:31.008808 10315 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:55:31.008920 10315 raft_consensus.cc:493] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cc4c5a10a6b842bf8696928b12c8848c)
I20260809 07:55:31.009030 10315 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:31.012344 10315 raft_consensus.cc:515] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } }
I20260809 07:55:31.013762 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafecff6ae1d4269848f56e43cd57594" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "cc4c5a10a6b842bf8696928b12c8848c"
I20260809 07:55:31.013975 10062 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:31.014623 10315 leader_election.cc:290] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [CANDIDATE]: Term 2 election: Requested vote from peers cc4c5a10a6b842bf8696928b12c8848c (127.26.37.62:36555), 0e58bc7a7bc147319d4b5db97d7935f0 (127.26.37.60:38395), 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59:38257)
I20260809 07:55:31.014859 9752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafecff6ae1d4269848f56e43cd57594" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9"
I20260809 07:55:31.015168 9752 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:31.017544 9307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dafecff6ae1d4269848f56e43cd57594" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "0e58bc7a7bc147319d4b5db97d7935f0"
I20260809 07:55:31.017762 9307 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:31.019789 9752 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafecff6ae1d4269848f56e43cd57594 in term 2.
I20260809 07:55:31.020851 9307 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafecff6ae1d4269848f56e43cd57594 in term 2.
I20260809 07:55:31.021426 9196 leader_election.cc:304] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0b0fc88f56a44864a2cbcd4613e4d6a9, 0e58bc7a7bc147319d4b5db97d7935f0, dafecff6ae1d4269848f56e43cd57594; no voters:
I20260809 07:55:31.021757 10315 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:55:31.024379 10319 leader_election.cc:290] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc4c5a10a6b842bf8696928b12c8848c (127.26.37.62:36555), dafecff6ae1d4269848f56e43cd57594 (127.26.37.61:41601), 0b0fc88f56a44864a2cbcd4613e4d6a9 (127.26.37.59:38257)
I20260809 07:55:31.024657 10062 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafecff6ae1d4269848f56e43cd57594 in term 2.
I20260809 07:55:31.024755 10315 raft_consensus.cc:697] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [term 2 LEADER]: Becoming Leader. State: Replica: dafecff6ae1d4269848f56e43cd57594, State: Running, Role: LEADER
I20260809 07:55:31.025611 10315 consensus_queue.cc:237] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } }
I20260809 07:55:31.026515 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "cc4c5a10a6b842bf8696928b12c8848c" is_pre_election: true
I20260809 07:55:31.026957 10062 raft_consensus.cc:2393] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e58bc7a7bc147319d4b5db97d7935f0 in current term 2: Already voted for candidate dafecff6ae1d4269848f56e43cd57594 in this term.
I20260809 07:55:31.028522 9752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" is_pre_election: true
I20260809 07:55:31.028811 9752 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e58bc7a7bc147319d4b5db97d7935f0 in current term 2: Already voted for candidate dafecff6ae1d4269848f56e43cd57594 in this term.
I20260809 07:55:31.028602 10324 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafecff6ae1d4269848f56e43cd57594. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.028942 10324 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:31.029287 9278 leader_election.cc:304] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e58bc7a7bc147319d4b5db97d7935f0, dafecff6ae1d4269848f56e43cd57594; no voters: 0b0fc88f56a44864a2cbcd4613e4d6a9, cc4c5a10a6b842bf8696928b12c8848c
I20260809 07:55:31.029834 10319 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:55:31.029913 10327 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:55:31.032827 10327 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:55:31.034025 10327 catalog_manager.cc:1295] Loaded cluster ID: 870aa0d0a4404e7f8f6691207a556320
I20260809 07:55:31.034121 10327 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:55:31.035966 10327 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:55:31.037195 10327 catalog_manager.cc:6103] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594: Loaded TSK: 0
I20260809 07:55:31.038350 10327 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260809 07:55:31.385203 10325 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:31.385524 10325 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:31.385604 10325 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:55:31.385663 10325 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:31.393738 10325 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:31.393929 10325 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:31.403913 10325 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34475
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42957
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:31.405659 10325 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:31.407480 10325 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:31.417168 10333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:31.417469 10334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:31.418411 10336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:31.418803 10325 server_base.cc:1061] running on GCE node
I20260809 07:55:31.419428 10325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:31.420253 10325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:31.421454 10325 hybrid_clock.cc:648] HybridClock initialized: now 1786262131421393 us; error 25 us; skew 500 ppm
I20260809 07:55:31.421761 10325 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:31.423954 10325 webserver.cc:533] Webserver started at http://127.26.37.1:42957/ using document root <none> and password file <none>
I20260809 07:55:31.424654 10325 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:31.424793 10325 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:31.430190 10325 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:55:31.432691 10343 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:31.434020 10325 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:31.434154 10325 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "157a5987150c4fdcbcf0dc9bb25bd25c"
format_stamp: "Formatted at 2026-08-09 07:55:27 on dist-test-slave-bw88"
I20260809 07:55:31.434861 10325 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:31.453126 10062 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [term 2 FOLLOWER]: Refusing update from remote peer dafecff6ae1d4269848f56e43cd57594: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:31.454066 10324 consensus_queue.cc:1048] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:31.457979 10349 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafecff6ae1d4269848f56e43cd57594. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.458268 10349 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:31.461901 9307 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [term 2 FOLLOWER]: Refusing update from remote peer dafecff6ae1d4269848f56e43cd57594: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:31.462540 10324 consensus_queue.cc:1048] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:31.464136 10319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafecff6ae1d4269848f56e43cd57594. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.464366 10319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:31.466674 9752 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [term 2 FOLLOWER]: Refusing update from remote peer dafecff6ae1d4269848f56e43cd57594: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:31.466789 10315 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.466992 10315 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:31.467198 10315 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.467413 10315 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:31.467535 10315 consensus_queue.cc:1048] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:31.468421 10319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.468681 10319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e58bc7a7bc147319d4b5db97d7935f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:31.469403 10349 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.469606 10349 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4c5a10a6b842bf8696928b12c8848c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:31.473785 10363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.474017 10363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:31.474184 10357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafecff6ae1d4269848f56e43cd57594. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.474454 10357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0fc88f56a44864a2cbcd4613e4d6a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:31.474611 10325 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:31.475448 10325 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:31.475685 10325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:31.476616 10325 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:31.478307 10325 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:31.478394 10325 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:31.478467 10325 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:31.478526 10325 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:55:31.478837 10325 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:55:31.478905 10325 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:55:31.479038 10325 fault_injection.cc:59] injecting fault for kudu (pid 10325): FLAGS_fault_before_start (process will exit)
I20260809 07:55:31.503685 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:34475
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395
--webserver_port=42957
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257 with env {}
I20260809 07:55:31.855319 10315 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "dafecff6ae1d4269848f56e43cd57594" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cc4c5a10a6b842bf8696928b12c8848c" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36555 } } peers { permanent_uuid: "dafecff6ae1d4269848f56e43cd57594" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 41601 } } peers { permanent_uuid: "0e58bc7a7bc147319d4b5db97d7935f0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 38395 } } peers { permanent_uuid: "0b0fc88f56a44864a2cbcd4613e4d6a9" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38257 } attrs { promote: false } } }
I20260809 07:55:31.855603 10315 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafecff6ae1d4269848f56e43cd57594 [sys.catalog]: This master's current role is: LEADER
W20260809 07:55:31.867473 10367 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:31.867843 10367 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:31.867990 10367 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:31.876209 10367 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:31.876413 10367 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:31.888907 10367 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:34475
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=42957
--tserver_master_addrs=127.26.37.62:36555,127.26.37.61:41601,127.26.37.60:38395,127.26.37.59:38257
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:31.890928 10367 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:31.893075 10367 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:31.903249 10374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:31.903280 10376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:31.903988 10379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:31.904505 10367 server_base.cc:1061] running on GCE node
I20260809 07:55:31.905107 10367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:31.906188 10367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:31.907419 10367 hybrid_clock.cc:648] HybridClock initialized: now 1786262131907382 us; error 47 us; skew 500 ppm
I20260809 07:55:31.907832 10367 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:31.910364 10367 webserver.cc:533] Webserver started at http://127.26.37.1:42957/ using document root <none> and password file <none>
I20260809 07:55:31.911147 10367 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:31.911293 10367 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:31.916975 10367 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:55:31.919713 10386 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:31.920989 10367 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:55:31.921231 10367 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "157a5987150c4fdcbcf0dc9bb25bd25c"
format_stamp: "Formatted at 2026-08-09 07:55:27 on dist-test-slave-bw88"
I20260809 07:55:31.922037 10367 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:31.943742 10367 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:31.944475 10367 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:31.944751 10367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:31.945672 10367 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:31.947319 10367 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:31.947427 10367 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:31.947582 10367 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:31.947678 10367 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:31.982645 10367 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34475
I20260809 07:55:31.982728 10540 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34475 every 8 connection(s)
I20260809 07:55:31.984122 10367 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:31.990123 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 10367
I20260809 07:55:31.990422 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 10367
I20260809 07:55:32.016098 10541 heartbeater.cc:344] Connected to a master server at 127.26.37.59:38257
I20260809 07:55:32.016614 10541 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:32.018108 10541 heartbeater.cc:507] Master 127.26.37.59:38257 requested a full tablet report, sending...
I20260809 07:55:32.020195 9741 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:32.022091 10547 heartbeater.cc:344] Connected to a master server at 127.26.37.61:41601
I20260809 07:55:32.022251 10547 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:32.022795 10547 heartbeater.cc:507] Master 127.26.37.61:41601 requested a full tablet report, sending...
I20260809 07:55:32.023717 9211 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
I20260809 07:55:32.025615 10535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:32.025766 9211 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:57875
I20260809 07:55:32.111150 10545 heartbeater.cc:344] Connected to a master server at 127.26.37.60:38395
I20260809 07:55:32.112056 10545 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:32.112797 10545 heartbeater.cc:507] Master 127.26.37.60:38395 requested a full tablet report, sending...
I20260809 07:55:32.113729 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 10367
I20260809 07:55:32.118521 9297 ts_manager.cc:194] Re-registered known tserver with Master: 157a5987150c4fdcbcf0dc9bb25bd25c (127.26.37.1:34475)
W20260809 07:55:32.141125 9280 connection.cc:573] server connection from 127.26.37.1:60867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:55:32.141613 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 10007
W20260809 07:55:32.142462 10560 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:55:32.009772 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:33443 (local address 127.26.37.62:36555)
0809 07:55:32.011495 (+ 1723us) server_negotiation.cc:207] Beginning negotiation
0809 07:55:32.011501 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:55:32.019828 (+ 8327us) server_negotiation.cc:408] Connection header received
0809 07:55:32.020076 (+ 248us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:55:32.020081 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:55:32.020176 (+ 95us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:55:32.020278 (+ 102us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:55:32.023314 (+ 3036us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:55:32.024737 (+ 1423us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:55:32.113410 (+ 88673us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:55:32.114028 (+ 618us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:55:32.141366 (+ 27338us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:33443: BlockingRecv error: recv got EOF from 127.26.37.1:33443 (error 108)
Metrics: {"server-negotiator.queue_time_us":1610,"thread_start_us":1505,"threads_started":1}
I20260809 07:55:32.148325 10089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:32.253679 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 10007
I20260809 07:55:32.281999 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9171
I20260809 07:55:32.283159 9247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:32.394742 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9171
I20260809 07:55:32.423599 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9258
I20260809 07:55:32.424768 9338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:32.532685 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9258
I20260809 07:55:32.561761 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 9575
I20260809 07:55:32.563128 9781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:32.676558 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 9575
Shutting down hive metastore.
2026-08-09T07:55:33Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/4 (23888 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/5
2026-08-09T07:55:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:55:33Z Disabled control of system clock
2026-08-09T07:55:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:55:33Z Disabled control of system clock
2026-08-09T07:55:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:55:33Z Disabled control of system clock
2026-08-09T07:55:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:55:33Z Disabled control of system clock
2026-08-09T07:55:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:55:33Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/security.properties
2026-08-09 07:55:40: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260809 07:55:49.410490 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39721
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401 with env {}
W20260809 07:55:49.760156 10938 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:49.760515 10938 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:49.760643 10938 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:49.768864 10938 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:49.768949 10938 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:49.769062 10938 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:49.769131 10938 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:49.778892 10938 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:49.780563 10938 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:49.782580 10938 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:49.791482 10945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:49.791515 10947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:49.791785 10950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:49.792660 10938 server_base.cc:1061] running on GCE node
I20260809 07:55:49.793427 10938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:49.795070 10938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:49.796262 10938 hybrid_clock.cc:648] HybridClock initialized: now 1786262149796133 us; error 100 us; skew 500 ppm
I20260809 07:55:49.796607 10938 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:49.798641 10938 webserver.cc:533] Webserver started at http://127.26.37.62:35173/ using document root <none> and password file <none>
I20260809 07:55:49.799356 10938 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:49.799487 10938 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:49.799862 10938 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:49.803376 10938 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "2ea42ac0f96a4be6a2389586082f3fc1"
format_stamp: "Formatted at 2026-08-09 07:55:49 on dist-test-slave-bw88"
I20260809 07:55:49.804188 10938 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "2ea42ac0f96a4be6a2389586082f3fc1"
format_stamp: "Formatted at 2026-08-09 07:55:49 on dist-test-slave-bw88"
I20260809 07:55:49.809469 10938 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260809 07:55:49.812672 10959 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:49.813979 10938 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:49.814153 10938 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "2ea42ac0f96a4be6a2389586082f3fc1"
format_stamp: "Formatted at 2026-08-09 07:55:49 on dist-test-slave-bw88"
I20260809 07:55:49.814352 10938 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:49.840754 10938 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:49.841652 10938 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:49.841922 10938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:49.862242 10938 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39721
I20260809 07:55:49.862267 11032 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39721 every 8 connection(s)
I20260809 07:55:49.863735 10938 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:55:49.865731 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 10938
I20260809 07:55:49.865928 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:55:49.866962 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38377
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401 with env {}
I20260809 07:55:49.868623 11034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:49.874531 11034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:49.891167 11034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:49.892280 10967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38377: connect: Connection refused (error 111)
W20260809 07:55:49.893942 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38377: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38377: connect: Connection refused (error 111)
I20260809 07:55:49.943274 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } attempt: 1
W20260809 07:55:49.945314 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38377: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38377: connect: Connection refused (error 111)
I20260809 07:55:50.013595 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } attempt: 2
W20260809 07:55:50.015784 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38377: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38377: connect: Connection refused (error 111)
I20260809 07:55:50.107097 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } attempt: 3
W20260809 07:55:50.109305 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38377: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38377: connect: Connection refused (error 111)
W20260809 07:55:50.227134 11038 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:50.227489 11038 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:50.227632 11038 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:50.235839 11038 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:50.235926 11038 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:50.236008 11038 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:50.236094 11038 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:50.245901 11038 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:50.247535 11038 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:50.249405 11038 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:55:50.252600 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } attempt: 4
W20260809 07:55:50.254742 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38377: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38377: connect: Connection refused (error 111)
W20260809 07:55:50.257822 11047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:50.258323 11046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:50.258723 11050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:50.259126 11038 server_base.cc:1061] running on GCE node
I20260809 07:55:50.259842 11038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:50.261441 11038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:50.262676 11038 hybrid_clock.cc:648] HybridClock initialized: now 1786262150262535 us; error 107 us; skew 500 ppm
I20260809 07:55:50.263041 11038 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:50.265110 11038 webserver.cc:533] Webserver started at http://127.26.37.61:40771/ using document root <none> and password file <none>
I20260809 07:55:50.265882 11038 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:50.266024 11038 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:50.266381 11038 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:50.268827 11038 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "b891814873234c21a8e8b4586c05d90a"
format_stamp: "Formatted at 2026-08-09 07:55:50 on dist-test-slave-bw88"
I20260809 07:55:50.269690 11038 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "b891814873234c21a8e8b4586c05d90a"
format_stamp: "Formatted at 2026-08-09 07:55:50 on dist-test-slave-bw88"
I20260809 07:55:50.274909 11038 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:55:50.278360 11058 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:50.279675 11038 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:55:50.279853 11038 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "b891814873234c21a8e8b4586c05d90a"
format_stamp: "Formatted at 2026-08-09 07:55:50 on dist-test-slave-bw88"
I20260809 07:55:50.280063 11038 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:50.299304 11038 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:50.300127 11038 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:50.300405 11038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:50.321173 11038 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:38377
I20260809 07:55:50.321190 11129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:38377 every 8 connection(s)
I20260809 07:55:50.322785 11038 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:55:50.325732 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11038
I20260809 07:55:50.325938 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:55:50.327306 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37401
--webserver_interface=127.26.37.60
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401 with env {}
I20260809 07:55:50.327632 11130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:50.336236 11130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:50.350984 11130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:50.360904 11130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:50.362397 11060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
W20260809 07:55:50.364104 11130 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
I20260809 07:55:50.413520 11130 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 1
W20260809 07:55:50.415609 11130 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
I20260809 07:55:50.483827 11130 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 2
W20260809 07:55:50.486009 11130 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
I20260809 07:55:50.554003 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } attempt: 5
I20260809 07:55:50.561026 11034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:50.562963 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
I20260809 07:55:50.577260 11130 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 3
W20260809 07:55:50.579349 11130 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
I20260809 07:55:50.614192 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 1
W20260809 07:55:50.616652 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
I20260809 07:55:50.684909 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 2
W20260809 07:55:50.687222 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
W20260809 07:55:50.700399 11134 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:50.700752 11134 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:50.700876 11134 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:50.709036 11134 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:50.709147 11134 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:50.709249 11134 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:50.709337 11134 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:50.718699 11134 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:37401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:50.720377 11134 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:50.722302 11134 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:55:50.722548 11130 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 4
W20260809 07:55:50.724502 11130 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
W20260809 07:55:50.730918 11146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:50.731015 11143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:50.731063 11142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:50.731839 11134 server_base.cc:1061] running on GCE node
I20260809 07:55:50.732522 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:50.734174 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:50.735399 11134 hybrid_clock.cc:648] HybridClock initialized: now 1786262150735273 us; error 94 us; skew 500 ppm
I20260809 07:55:50.735749 11134 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:50.737910 11134 webserver.cc:533] Webserver started at http://127.26.37.60:38969/ using document root <none> and password file <none>
I20260809 07:55:50.738646 11134 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:50.738778 11134 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:50.739137 11134 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:50.741514 11134 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d"
format_stamp: "Formatted at 2026-08-09 07:55:50 on dist-test-slave-bw88"
I20260809 07:55:50.742311 11134 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d"
format_stamp: "Formatted at 2026-08-09 07:55:50 on dist-test-slave-bw88"
I20260809 07:55:50.748440 11134 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 07:55:50.751777 11153 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:50.753036 11134 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:55:50.753204 11134 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d"
format_stamp: "Formatted at 2026-08-09 07:55:50 on dist-test-slave-bw88"
I20260809 07:55:50.753444 11134 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:50.778712 11134 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:50.779593 11134 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:50.779896 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:50.793426 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 3
W20260809 07:55:50.795590 11034 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:37401: Network error: Client connection negotiation failed: client connection to 127.26.37.60:37401: connect: Connection refused (error 111)
I20260809 07:55:50.800755 11134 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:37401
I20260809 07:55:50.800762 11218 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:37401 every 8 connection(s)
I20260809 07:55:50.802351 11134 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:55:50.806581 11219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:50.808753 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11134
I20260809 07:55:50.808943 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:55:50.813704 11219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:50.827994 11219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:50.833581 11219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:50.843158 11219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: Bootstrap starting.
I20260809 07:55:50.846421 11219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:50.847617 11219 log.cc:826] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:50.850502 11219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: No bootstrap required, opened a new log
I20260809 07:55:50.856205 11219 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:50.856483 11219 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:50.856611 11219 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c39c13c93ce4cd9b640d6b18c3ed10d, State: Initialized, Role: FOLLOWER
I20260809 07:55:50.857227 11219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:50.858191 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:50.858443 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:50.859288 11219 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:55:50.870330 11243 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:50.870515 11243 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:50.867519 11134 master_runner.cc:187] Error getting master registration for 127.26.37.62:39721: OK, instance_id { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" instance_seqno: 1786262149856721 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:55:50.874012 11134 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:55:50.972877 11034 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 4
I20260809 07:55:50.983042 11034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Bootstrap starting.
I20260809 07:55:50.986868 11034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:50.987986 11034 log.cc:826] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:50.990811 11034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: No bootstrap required, opened a new log
I20260809 07:55:50.996196 11034 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:50.996447 11034 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:50.996608 11034 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea42ac0f96a4be6a2389586082f3fc1, State: Initialized, Role: FOLLOWER
I20260809 07:55:50.997267 11034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:50.998371 11246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:50.998703 11246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:50.999511 11034 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:55:51.011416 11257 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:51.011555 11257 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:51.010603 10938 master_runner.cc:187] Error getting master registration for 127.26.37.61:38377: OK, instance_id { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" instance_seqno: 1786262150315661 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:55:51.014991 10938 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:55:51.023730 11130 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } attempt: 5
W20260809 07:55:51.031980 11161 tablet.cc:2406] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:55:51.036510 11130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: Bootstrap starting.
I20260809 07:55:51.039749 11130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:51.040798 11130 log.cc:826] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:51.044725 11130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: No bootstrap required, opened a new log
I20260809 07:55:51.052515 11130 raft_consensus.cc:359] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:51.052852 11130 raft_consensus.cc:385] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:51.052984 11130 raft_consensus.cc:740] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b891814873234c21a8e8b4586c05d90a, State: Initialized, Role: FOLLOWER
I20260809 07:55:51.053718 11130 consensus_queue.cc:260] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:51.054744 11261 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:51.055109 11261 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:51.055986 11130 sys_catalog.cc:565] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:51.074429 11246 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:55:51.074698 11246 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:51.076396 11246 leader_election.cc:290] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b891814873234c21a8e8b4586c05d90a (127.26.37.61:38377), 4c39c13c93ce4cd9b640d6b18c3ed10d (127.26.37.60:37401)
I20260809 07:55:51.077005 11094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b891814873234c21a8e8b4586c05d90a" is_pre_election: true
I20260809 07:55:51.077527 11094 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea42ac0f96a4be6a2389586082f3fc1 in term 0.
I20260809 07:55:51.078157 10967 leader_election.cc:304] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea42ac0f96a4be6a2389586082f3fc1, b891814873234c21a8e8b4586c05d90a; no voters:
I20260809 07:55:51.079258 11189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" is_pre_election: true
I20260809 07:55:51.079762 11189 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea42ac0f96a4be6a2389586082f3fc1 in term 0.
I20260809 07:55:51.080754 11038 master_runner.cc:264] No leader master found from master b891814873234c21a8e8b4586c05d90a
I20260809 07:55:51.080873 11038 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:55:51.081056 11246 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:55:51.081231 11246 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:55:51.081362 11246 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:55:51.085155 11246 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:51.086427 11094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b891814873234c21a8e8b4586c05d90a"
I20260809 07:55:51.086652 11094 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:55:51.087494 11274 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:51.087680 11274 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 07:55:51.094779 10969 tablet.cc:2406] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:55:51.095535 11094 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea42ac0f96a4be6a2389586082f3fc1 in term 1.
I20260809 07:55:51.097105 10967 leader_election.cc:304] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea42ac0f96a4be6a2389586082f3fc1, b891814873234c21a8e8b4586c05d90a; no voters:
I20260809 07:55:51.099836 11246 leader_election.cc:290] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [CANDIDATE]: Term 1 election: Requested vote from peers b891814873234c21a8e8b4586c05d90a (127.26.37.61:38377), 4c39c13c93ce4cd9b640d6b18c3ed10d (127.26.37.60:37401)
I20260809 07:55:51.100126 11246 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:55:51.100291 11246 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea42ac0f96a4be6a2389586082f3fc1, State: Running, Role: LEADER
I20260809 07:55:51.100849 11246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } }
I20260809 07:55:51.103317 11189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d"
I20260809 07:55:51.103538 11189 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:55:51.107090 11189 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea42ac0f96a4be6a2389586082f3fc1 in term 1.
I20260809 07:55:51.113102 11277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea42ac0f96a4be6a2389586082f3fc1. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:51.113477 11277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:51.122022 11280 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:55:51.124413 11280 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:55:51.133711 11189 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 1 FOLLOWER]: Refusing update from remote peer 2ea42ac0f96a4be6a2389586082f3fc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:55:51.134750 11277 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:55:51.142513 11094 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Refusing update from remote peer 2ea42ac0f96a4be6a2389586082f3fc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:55:51.143483 11277 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:55:51.180229 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea42ac0f96a4be6a2389586082f3fc1. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:51.180527 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:51.185153 11261 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea42ac0f96a4be6a2389586082f3fc1. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:51.185452 11261 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:51.186427 11281 mvcc.cc:204] Tried to move back new op lower bound from 7316529771031183360 to 7316529770915770368. Current Snapshot: MvccSnapshot[applied={T|T < 7316529771031183360}]
I20260809 07:55:51.193172 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:51.193398 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:51.194821 11261 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:51.195036 11261 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:51.202023 11280 catalog_manager.cc:1383] Generated new cluster ID: 89cf957702214ed7be608c50010a9500
I20260809 07:55:51.202188 11280 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:55:51.226163 11246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:51.226483 11246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:51.226758 11277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } }
I20260809 07:55:51.226958 11277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:51.248304 11280 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:55:51.250120 11280 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:55:51.273592 11280 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Generated new TSK 0
I20260809 07:55:51.274740 11280 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:55:51.282495 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260809 07:55:51.679584 11299 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:51.679926 11299 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:51.680075 11299 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:51.688560 11299 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:51.688761 11299 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:51.698612 11299 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:51.700266 11299 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:51.702225 11299 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:51.712613 11305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:51.719650 11304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:51.720333 11307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:51.721208 11299 server_base.cc:1061] running on GCE node
I20260809 07:55:51.721876 11299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:51.725627 11299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:51.726894 11299 hybrid_clock.cc:648] HybridClock initialized: now 1786262151726756 us; error 112 us; skew 500 ppm
I20260809 07:55:51.727259 11299 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:51.729480 11299 webserver.cc:533] Webserver started at http://127.26.37.1:41463/ using document root <none> and password file <none>
I20260809 07:55:51.730244 11299 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:51.730371 11299 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:51.730737 11299 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:51.732985 11299 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "f7724a564c8240378a695d2d0c5029d6"
format_stamp: "Formatted at 2026-08-09 07:55:51 on dist-test-slave-bw88"
I20260809 07:55:51.733810 11299 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "f7724a564c8240378a695d2d0c5029d6"
format_stamp: "Formatted at 2026-08-09 07:55:51 on dist-test-slave-bw88"
I20260809 07:55:51.739213 11299 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:55:51.742599 11313 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:51.743892 11299 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:51.744059 11299 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f7724a564c8240378a695d2d0c5029d6"
format_stamp: "Formatted at 2026-08-09 07:55:51 on dist-test-slave-bw88"
I20260809 07:55:51.744270 11299 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:51.773680 11299 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:51.774587 11299 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:51.774853 11299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:51.775782 11299 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:51.777562 11299 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:51.777683 11299 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:51.777824 11299 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:51.777904 11299 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:51.818038 11299 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39287
I20260809 07:55:51.818120 11466 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39287 every 8 connection(s)
I20260809 07:55:51.819624 11299 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:51.830108 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11299
I20260809 07:55:51.830341 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:55:51.843612 11472 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38377
I20260809 07:55:51.843647 11469 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37401
I20260809 07:55:51.843865 11471 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39721
I20260809 07:55:51.844115 11472 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:51.844112 11471 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:51.844112 11469 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:51.845206 11471 heartbeater.cc:507] Master 127.26.37.62:39721 requested a full tablet report, sending...
I20260809 07:55:51.845244 11472 heartbeater.cc:507] Master 127.26.37.61:38377 requested a full tablet report, sending...
I20260809 07:55:51.845293 11469 heartbeater.cc:507] Master 127.26.37.60:37401 requested a full tablet report, sending...
I20260809 07:55:51.847451 11175 ts_manager.cc:194] Registered new tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:51.847456 11081 ts_manager.cc:194] Registered new tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:51.848763 10985 ts_manager.cc:194] Registered new tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:51.850533 10985 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:48519
I20260809 07:55:51.856945 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:51.859541 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:37057
--webserver_interface=127.26.37.59
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057 with env {}
I20260809 07:55:51.873023 11243 catalog_manager.cc:1295] Loaded cluster ID: 89cf957702214ed7be608c50010a9500
I20260809 07:55:51.873134 11243 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: loading cluster ID for follower catalog manager: success
I20260809 07:55:51.876569 11243 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: acquiring CA information for follower catalog manager: success
I20260809 07:55:51.878443 11243 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:55:52.090620 11274 catalog_manager.cc:1295] Loaded cluster ID: 89cf957702214ed7be608c50010a9500
I20260809 07:55:52.090764 11274 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: loading cluster ID for follower catalog manager: success
I20260809 07:55:52.094853 11274 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: acquiring CA information for follower catalog manager: success
I20260809 07:55:52.097271 11274 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:55:52.221750 11485 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:52.222054 11485 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:52.222127 11485 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:52.230271 11485 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:52.230346 11485 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:52.230399 11485 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:52.230445 11485 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:52.240118 11485 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:37057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:52.241650 11485 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:52.243492 11485 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:52.252413 11492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:52.252467 11490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:52.253062 11501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:52.253526 11485 server_base.cc:1061] running on GCE node
I20260809 07:55:52.254235 11485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:52.255699 11485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:52.256925 11485 hybrid_clock.cc:648] HybridClock initialized: now 1786262152256766 us; error 128 us; skew 500 ppm
I20260809 07:55:52.257273 11485 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:52.259553 11485 webserver.cc:533] Webserver started at http://127.26.37.59:40325/ using document root <none> and password file <none>
I20260809 07:55:52.260413 11485 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:52.260550 11485 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:52.260952 11485 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:52.263681 11485 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "ecc70316b54f4e588252a8dea3c7fceb"
format_stamp: "Formatted at 2026-08-09 07:55:52 on dist-test-slave-bw88"
I20260809 07:55:52.264477 11485 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "ecc70316b54f4e588252a8dea3c7fceb"
format_stamp: "Formatted at 2026-08-09 07:55:52 on dist-test-slave-bw88"
I20260809 07:55:52.270078 11485 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:55:52.273607 11507 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:52.274917 11485 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:55:52.275094 11485 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "ecc70316b54f4e588252a8dea3c7fceb"
format_stamp: "Formatted at 2026-08-09 07:55:52 on dist-test-slave-bw88"
I20260809 07:55:52.275319 11485 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:52.299327 11485 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:52.300156 11485 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:52.300432 11485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:52.321079 11485 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:37057
I20260809 07:55:52.321098 11575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:37057 every 8 connection(s)
I20260809 07:55:52.322957 11485 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:55:52.327342 11576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:52.327512 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11485
I20260809 07:55:52.336162 11576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:52.347220 11576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:52.353212 11576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:52.359220 11576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:55:52.368561 11576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Bootstrap starting.
I20260809 07:55:52.371718 11576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Neither blocks nor log segments found. Creating new log.
I20260809 07:55:52.372855 11576 log.cc:826] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:52.375289 11576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: No bootstrap required, opened a new log
I20260809 07:55:52.380344 11576 raft_consensus.cc:359] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } }
I20260809 07:55:52.380669 11576 raft_consensus.cc:385] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:55:52.380776 11576 raft_consensus.cc:740] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecc70316b54f4e588252a8dea3c7fceb, State: Initialized, Role: FOLLOWER
I20260809 07:55:52.381428 11576 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } }
I20260809 07:55:52.382277 11590 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } } }
I20260809 07:55:52.382561 11590 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:52.383227 11576 sys_catalog.cc:565] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:55:52.392304 11606 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:55:52.392455 11606 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:55:52.396137 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11299
I20260809 07:55:52.400363 11485 master_runner.cc:429] Detected that this master ecc70316b54f4e588252a8dea3c7fceb is joining an existing cluster
I20260809 07:55:52.400465 11485 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:55:52.401443 11485 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:55:52.401765 11485 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:55:52.401978 11485 tablet_replica.cc:333] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: stopping tablet replica
I20260809 07:55:52.404222 11485 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:37057
I20260809 07:55:52.406137 11461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:52.453042 10985 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:37057: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 37057
}
attrs {
promote: true
}
}
dest_uuid: "2ea42ac0f96a4be6a2389586082f3fc1"
cas_config_opid_index: -1
I20260809 07:55:52.454734 10985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:52.458693 11094 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Refusing update from remote peer 2ea42ac0f96a4be6a2389586082f3fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260809 07:55:52.458842 10964 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 -> Peer ecc70316b54f4e588252a8dea3c7fceb (127.26.37.59:37057): Couldn't send request to peer ecc70316b54f4e588252a8dea3c7fceb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:55:52.459758 11189 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 1 FOLLOWER]: Refusing update from remote peer 2ea42ac0f96a4be6a2389586082f3fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:55:52.460598 11432 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:55:52.461109 11432 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:55:52.470357 11432 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ecc70316b54f4e588252a8dea3c7fceb (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.475607 11189 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ecc70316b54f4e588252a8dea3c7fceb (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.481580 11432 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:37057
I20260809 07:55:52.482111 11432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.482388 11485 master.cc:562] Master@127.26.37.59:37057 shutting down...
I20260809 07:55:52.482545 11432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:52.483153 11624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.483443 11624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:52.483841 11628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.485184 11628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:52.496236 11624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.496465 11624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:52.497606 11092 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ecc70316b54f4e588252a8dea3c7fceb (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.501104 11631 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.501524 11631 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:52.529724 11485 master.cc:584] Master@127.26.37.59:37057 shutdown complete.
I20260809 07:55:52.529852 11485 master_runner.cc:306] Clearing existing system tablet
I20260809 07:55:52.530303 11485 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:52.530441 11485 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:52.533183 11485 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:55:52.534807 11635 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:52.535156 11485 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:52.535579 11485 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 07:55:52.535729 11485 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "ecc70316b54f4e588252a8dea3c7fceb"
format_stamp: "Formatted at 2026-08-09 07:55:52 on dist-test-slave-bw88"
I20260809 07:55:52.539387 11485 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:55:52.546114 11485 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:55:52.546288 11485 log.cc:1199] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:55:52.546778 11485 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Deleting consensus metadata
I20260809 07:55:52.547048 11485 master_runner.cc:316] Copying system tablet from 127.26.37.62:39721
I20260809 07:55:52.550088 11485 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.62:39721
I20260809 07:55:52.557982 11012 tablet_copy_service.cc:140] P 2ea42ac0f96a4be6a2389586082f3fc1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ecc70316b54f4e588252a8dea3c7fceb ({username='slave'} at 127.0.0.1:38014)
I20260809 07:55:52.558337 11012 tablet_copy_service.cc:161] P 2ea42ac0f96a4be6a2389586082f3fc1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ecc70316b54f4e588252a8dea3c7fceb at {username='slave'} at 127.0.0.1:38014: session id = ecc70316b54f4e588252a8dea3c7fceb-00000000000000000000000000000000
I20260809 07:55:52.561844 11012 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:55:52.564014 11485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:55:52.569401 11485 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: tablet copy: Starting download of 0 data blocks...
I20260809 07:55:52.569711 11485 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: tablet copy: Starting download of 1 WAL segments...
I20260809 07:55:52.572374 11485 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:55:52.576622 11012 tablet_copy_service.cc:342] P 2ea42ac0f96a4be6a2389586082f3fc1: Request end of tablet copy session ecc70316b54f4e588252a8dea3c7fceb-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:38014
I20260809 07:55:52.576855 11012 tablet_copy_service.cc:434] P 2ea42ac0f96a4be6a2389586082f3fc1: ending tablet copy session ecc70316b54f4e588252a8dea3c7fceb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ecc70316b54f4e588252a8dea3c7fceb
I20260809 07:55:52.577399 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11299
W20260809 07:55:52.587293 11485 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260809 07:55:52.591871 11485 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:52.595691 11645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:52.595759 11646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:52.603312 11648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:52.603812 11485 server_base.cc:1061] running on GCE node
I20260809 07:55:52.604158 11485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:52.605127 11485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:52.606060 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39287
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--webserver_port=41463
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057 with env {}
I20260809 07:55:52.606391 11485 hybrid_clock.cc:648] HybridClock initialized: now 1786262152606230 us; error 103 us; skew 500 ppm
I20260809 07:55:52.606642 11485 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:52.608203 11485 webserver.cc:533] Webserver started at http://127.26.37.59:42503/ using document root <none> and password file <none>
I20260809 07:55:52.608598 11485 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:52.608723 11485 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:52.611182 11485 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 07:55:52.619041 11654 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:52.619539 11485 fs_manager.cc:730] Time spent opening block manager: real 0.007s user 0.004s sys 0.004s
I20260809 07:55:52.619750 11485 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "ecc70316b54f4e588252a8dea3c7fceb"
format_stamp: "Formatted at 2026-08-09 07:55:52 on dist-test-slave-bw88"
I20260809 07:55:52.619930 11485 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:52.666452 11485 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:52.667392 11485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:52.691913 11485 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:37057
I20260809 07:55:52.691963 11729 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:37057 every 8 connection(s)
I20260809 07:55:52.693213 11485 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:55:52.697928 11732 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:55:52.701651 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Bootstrap starting.
I20260809 07:55:52.718088 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:55:52.718652 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: Bootstrap complete.
I20260809 07:55:52.719862 11732 raft_consensus.cc:359] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:52.720099 11732 raft_consensus.cc:740] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecc70316b54f4e588252a8dea3c7fceb, State: Initialized, Role: LEARNER
I20260809 07:55:52.720362 11732 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:52.721460 11737 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:52.721800 11737 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:52.721843 11732 sys_catalog.cc:565] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:52.732477 11750 catalog_manager.cc:1295] Loaded cluster ID: 89cf957702214ed7be608c50010a9500
I20260809 07:55:52.732963 11750 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: loading cluster ID for follower catalog manager: success
I20260809 07:55:52.737010 11750 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: acquiring CA information for follower catalog manager: success
I20260809 07:55:52.739531 11750 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:55:53.014783 11657 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:53.015082 11657 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:53.015226 11657 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:53.023546 11657 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:53.023741 11657 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:53.033592 11657 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39287
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41463
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:53.035233 11657 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:53.037181 11657 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:53.047905 11759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:53.047854 11757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:53.049707 11761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:53.050659 11657 server_base.cc:1061] running on GCE node
I20260809 07:55:53.051296 11657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:53.052776 11657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:53.054239 11698 raft_consensus.cc:1217] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:55:53.054876 11737 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea42ac0f96a4be6a2389586082f3fc1. Latest consensus state: current_term: 1 leader_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:53.055140 11737 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:53.056118 11657 hybrid_clock.cc:648] HybridClock initialized: now 1786262153055914 us; error 167 us; skew 500 ppm
I20260809 07:55:53.056404 11657 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:53.059127 11657 webserver.cc:533] Webserver started at http://127.26.37.1:41463/ using document root <none> and password file <none>
I20260809 07:55:53.060114 11657 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:53.060282 11657 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:53.065959 11657 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:55:53.068691 11768 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:53.070029 11657 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:55:53.070201 11657 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f7724a564c8240378a695d2d0c5029d6"
format_stamp: "Formatted at 2026-08-09 07:55:51 on dist-test-slave-bw88"
I20260809 07:55:53.070889 11657 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:53.091085 11657 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:53.092242 11657 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:53.092679 11657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:53.093986 11657 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:53.097826 11657 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:53.097927 11657 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:53.098071 11657 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:53.098147 11657 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:53.142207 11657 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39287
I20260809 07:55:53.142256 11917 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39287 every 8 connection(s)
I20260809 07:55:53.143859 11657 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:53.152971 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11657
I20260809 07:55:53.164763 11919 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37401
I20260809 07:55:53.164821 11923 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38377
I20260809 07:55:53.165158 11919 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:53.165175 11923 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:53.166541 11923 heartbeater.cc:507] Master 127.26.37.61:38377 requested a full tablet report, sending...
I20260809 07:55:53.166579 11919 heartbeater.cc:507] Master 127.26.37.60:37401 requested a full tablet report, sending...
I20260809 07:55:53.168840 11175 ts_manager.cc:194] Re-registered known tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:53.169076 11081 ts_manager.cc:194] Re-registered known tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:53.169456 11918 heartbeater.cc:344] Connected to a master server at 127.26.37.59:37057
I20260809 07:55:53.169598 11918 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:53.170110 11918 heartbeater.cc:507] Master 127.26.37.59:37057 requested a full tablet report, sending...
I20260809 07:55:53.172286 11682 ts_manager.cc:194] Registered new tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:53.173882 11921 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39721
I20260809 07:55:53.174021 11921 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:53.174453 11921 heartbeater.cc:507] Master 127.26.37.62:39721 requested a full tablet report, sending...
I20260809 07:55:53.175423 10985 ts_manager.cc:194] Re-registered known tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:53.176510 10985 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:49835
I20260809 07:55:53.179694 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:53.187397 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:55:53.187557 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 10938
I20260809 07:55:53.188359 11028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:53.299319 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 10938
I20260809 07:55:53.329058 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39721
--webserver_interface=127.26.37.62
--webserver_port=35173
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057 with env {}
W20260809 07:55:53.689320 11941 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:53.689670 11941 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:53.689752 11941 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:53.697855 11941 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:53.697928 11941 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:53.698011 11941 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:53.698091 11941 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:53.707649 11941 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:39721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=35173
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:53.709133 11941 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:53.711004 11941 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:53.719411 11949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:53.719656 11947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:53.719627 11951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:53.720306 11941 server_base.cc:1061] running on GCE node
I20260809 07:55:53.721060 11941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:53.722579 11941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:53.723773 11941 hybrid_clock.cc:648] HybridClock initialized: now 1786262153723654 us; error 91 us; skew 500 ppm
I20260809 07:55:53.724092 11941 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:53.726022 11941 webserver.cc:533] Webserver started at http://127.26.37.62:35173/ using document root <none> and password file <none>
I20260809 07:55:53.726758 11941 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:53.726918 11941 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:53.732012 11941 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 07:55:53.734768 11963 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:53.736073 11941 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:55:53.736232 11941 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "2ea42ac0f96a4be6a2389586082f3fc1"
format_stamp: "Formatted at 2026-08-09 07:55:49 on dist-test-slave-bw88"
I20260809 07:55:53.736845 11941 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:53.747895 11941 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:53.748642 11941 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:53.748924 11941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:53.770533 11941 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:39721
I20260809 07:55:53.770558 12037 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:39721 every 8 connection(s)
I20260809 07:55:53.772143 11941 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:55:53.774303 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 11941
I20260809 07:55:53.774616 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11657
I20260809 07:55:53.780314 12038 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:55:53.783890 11911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:53.784866 12038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Bootstrap starting.
I20260809 07:55:53.804184 12038 log.cc:826] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:55:53.815156 12038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:55:53.815718 12038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: Bootstrap complete.
I20260809 07:55:53.822523 12038 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:53.823311 12038 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea42ac0f96a4be6a2389586082f3fc1, State: Initialized, Role: FOLLOWER
I20260809 07:55:53.824131 12038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:53.825014 12050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:53.825330 12050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:53.826000 12038 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:55:53.836791 12067 catalog_manager.cc:1295] Loaded cluster ID: 89cf957702214ed7be608c50010a9500
I20260809 07:55:53.836885 12067 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: loading cluster ID for follower catalog manager: success
I20260809 07:55:53.840958 12067 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: acquiring CA information for follower catalog manager: success
I20260809 07:55:53.843376 12067 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:55:53.885056 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11657
I20260809 07:55:53.907951 11941 master_runner.cc:264] No leader master found from master 2ea42ac0f96a4be6a2389586082f3fc1
I20260809 07:55:53.908143 11941 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:55:53.911369 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39287
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--webserver_port=41463
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057 with env {}
W20260809 07:55:54.273620 12079 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:54.273909 12079 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:54.274040 12079 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:54.282092 12079 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:54.282294 12079 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:54.292795 12079 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39287
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41463
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:54.294425 12079 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:54.296212 12079 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:54.306058 12086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:54.306222 12088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:54.306357 12084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:54.307250 12079 server_base.cc:1061] running on GCE node
I20260809 07:55:54.307792 12079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:54.309008 12079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:54.310232 12079 hybrid_clock.cc:648] HybridClock initialized: now 1786262154310081 us; error 121 us; skew 500 ppm
I20260809 07:55:54.310545 12079 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:54.312584 12079 webserver.cc:533] Webserver started at http://127.26.37.1:41463/ using document root <none> and password file <none>
I20260809 07:55:54.313406 12079 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:54.313611 12079 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:54.318853 12079 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:55:54.321662 12095 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:54.323103 12079 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:54.323343 12079 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f7724a564c8240378a695d2d0c5029d6"
format_stamp: "Formatted at 2026-08-09 07:55:51 on dist-test-slave-bw88"
I20260809 07:55:54.324020 12079 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:54.356017 12079 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:54.357012 12079 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:54.357311 12079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:54.358422 12079 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:54.360333 12079 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:54.360414 12079 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:54.360565 12079 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:54.360616 12079 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:54.396016 12079 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39287
I20260809 07:55:54.396090 12254 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39287 every 8 connection(s)
I20260809 07:55:54.397636 12079 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:54.408133 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12079
I20260809 07:55:54.442023 12258 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38377
I20260809 07:55:54.442370 12258 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:54.443501 12258 heartbeater.cc:507] Master 127.26.37.61:38377 requested a full tablet report, sending...
I20260809 07:55:54.445601 11081 ts_manager.cc:194] Re-registered known tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:54.448746 12256 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37401
I20260809 07:55:54.448966 12256 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:54.449551 12256 heartbeater.cc:507] Master 127.26.37.60:37401 requested a full tablet report, sending...
I20260809 07:55:54.450778 11175 ts_manager.cc:194] Re-registered known tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:54.452819 12257 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39721
I20260809 07:55:54.452944 12257 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:54.453441 12257 heartbeater.cc:507] Master 127.26.37.62:39721 requested a full tablet report, sending...
I20260809 07:55:54.455261 11989 ts_manager.cc:194] Registered new tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:54.458571 12255 heartbeater.cc:344] Connected to a master server at 127.26.37.59:37057
I20260809 07:55:54.458724 12255 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:54.459203 12255 heartbeater.cc:507] Master 127.26.37.59:37057 requested a full tablet report, sending...
I20260809 07:55:54.460700 11682 ts_manager.cc:194] Re-registered known tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:54.468313 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12079
I20260809 07:55:54.474121 12275 raft_consensus.cc:493] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ea42ac0f96a4be6a2389586082f3fc1)
I20260809 07:55:54.474406 12275 raft_consensus.cc:515] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:54.476706 11189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b891814873234c21a8e8b4586c05d90a" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" is_pre_election: true
I20260809 07:55:54.477257 12275 leader_election.cc:290] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea42ac0f96a4be6a2389586082f3fc1 (127.26.37.62:39721), 4c39c13c93ce4cd9b640d6b18c3ed10d (127.26.37.60:37401)
I20260809 07:55:54.480721 12248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:54.484827 12003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b891814873234c21a8e8b4586c05d90a" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" is_pre_election: true
I20260809 07:55:54.485512 12003 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b891814873234c21a8e8b4586c05d90a in term 1.
I20260809 07:55:54.486176 11063 leader_election.cc:304] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ea42ac0f96a4be6a2389586082f3fc1, b891814873234c21a8e8b4586c05d90a; no voters: 4c39c13c93ce4cd9b640d6b18c3ed10d
I20260809 07:55:54.486725 12275 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:55:54.486840 12275 raft_consensus.cc:493] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ea42ac0f96a4be6a2389586082f3fc1)
I20260809 07:55:54.486929 12275 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:54.490160 12275 raft_consensus.cc:515] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:54.490929 12275 leader_election.cc:290] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [CANDIDATE]: Term 2 election: Requested vote from peers 2ea42ac0f96a4be6a2389586082f3fc1 (127.26.37.62:39721), 4c39c13c93ce4cd9b640d6b18c3ed10d (127.26.37.60:37401)
I20260809 07:55:54.491452 12003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b891814873234c21a8e8b4586c05d90a" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "2ea42ac0f96a4be6a2389586082f3fc1"
I20260809 07:55:54.491447 11189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b891814873234c21a8e8b4586c05d90a" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d"
I20260809 07:55:54.491675 12003 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:54.495756 12003 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b891814873234c21a8e8b4586c05d90a in term 2.
I20260809 07:55:54.496223 11063 leader_election.cc:304] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ea42ac0f96a4be6a2389586082f3fc1, b891814873234c21a8e8b4586c05d90a; no voters: 4c39c13c93ce4cd9b640d6b18c3ed10d
I20260809 07:55:54.496570 12275 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:55:54.497135 12275 raft_consensus.cc:697] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 2 LEADER]: Becoming Leader. State: Replica: b891814873234c21a8e8b4586c05d90a, State: Running, Role: LEADER
I20260809 07:55:54.497676 12275 consensus_queue.cc:237] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:54.499946 12280 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b891814873234c21a8e8b4586c05d90a. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:54.500231 12280 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:54.501132 12282 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:55:54.503310 12282 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:55:54.504153 12282 catalog_manager.cc:1295] Loaded cluster ID: 89cf957702214ed7be608c50010a9500
I20260809 07:55:54.504266 12282 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:55:54.505577 12282 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:55:54.506410 12282 catalog_manager.cc:6103] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: Loaded TSK: 0
I20260809 07:55:54.507265 12282 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:55:54.561350 12283 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ea42ac0f96a4be6a2389586082f3fc1)
I20260809 07:55:54.561623 12283 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } }
I20260809 07:55:54.563287 12283 leader_election.cc:290] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea42ac0f96a4be6a2389586082f3fc1 (127.26.37.62:39721), b891814873234c21a8e8b4586c05d90a (127.26.37.61:38377)
I20260809 07:55:54.564473 11092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "b891814873234c21a8e8b4586c05d90a" is_pre_election: true
I20260809 07:55:54.569981 12003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" is_pre_election: true
I20260809 07:55:54.570336 12003 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4c39c13c93ce4cd9b640d6b18c3ed10d in current term 2: Already voted for candidate b891814873234c21a8e8b4586c05d90a in this term.
I20260809 07:55:54.571025 11158 leader_election.cc:304] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c39c13c93ce4cd9b640d6b18c3ed10d; no voters: 2ea42ac0f96a4be6a2389586082f3fc1, b891814873234c21a8e8b4586c05d90a
I20260809 07:55:54.571388 12283 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:55:54.573640 12283 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:55:54.590687 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12079
I20260809 07:55:54.615574 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39287
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--webserver_port=41463
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057
--fault_before_start=1.0 with env {}
I20260809 07:55:54.930192 11698 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 1 LEARNER]: Advancing to term 2
I20260809 07:55:54.932719 11698 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 2 LEARNER]: Refusing update from remote peer b891814873234c21a8e8b4586c05d90a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 07:55:54.933570 12280 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:55:54.936944 12288 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b891814873234c21a8e8b4586c05d90a. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:54.937300 12288 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:54.937724 12275 raft_consensus.cc:1064] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: attempting to promote NON_VOTER ecc70316b54f4e588252a8dea3c7fceb to VOTER
W20260809 07:55:54.938025 12275 raft_consensus.cc:1068] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: Unable to promote non-voter ecc70316b54f4e588252a8dea3c7fceb: Illegal state: Leader has not yet committed an operation in its own term
W20260809 07:55:54.977864 12286 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:54.978227 12286 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:54.978358 12286 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:55:54.978447 12286 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:54.987205 12286 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:54.987442 12286 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:54.996297 11189 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 2 FOLLOWER]: Refusing update from remote peer b891814873234c21a8e8b4586c05d90a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 07:55:54.997013 12275 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:55:54.997774 12286 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39287
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41463
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:54.998881 12283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b891814873234c21a8e8b4586c05d90a. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:54.999151 12283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:54.999733 12286 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:55.001597 12280 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:55.001843 12280 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:55.002182 12280 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:55.002276 12286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:55:55.002401 12280 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:55.002848 12003 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 2 FOLLOWER]: Refusing update from remote peer b891814873234c21a8e8b4586c05d90a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260809 07:55:55.003865 12280 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:55:55.003938 12283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:55.004146 12283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:55.005743 12288 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:55.005985 12288 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: This master's current role is: LEARNER
I20260809 07:55:55.006904 12280 raft_consensus.cc:1064] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a: attempting to promote NON_VOTER ecc70316b54f4e588252a8dea3c7fceb to VOTER
I20260809 07:55:55.008215 12280 consensus_queue.cc:237] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } }
I20260809 07:55:55.009294 12305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:55.009764 12305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:55.010207 12300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b891814873234c21a8e8b4586c05d90a. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: true } } }
I20260809 07:55:55.010502 12300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:55.010878 11189 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 2 FOLLOWER]: Refusing update from remote peer b891814873234c21a8e8b4586c05d90a: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:55.011180 12003 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 2 FOLLOWER]: Refusing update from remote peer b891814873234c21a8e8b4586c05d90a: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:55.011713 12275 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:55.011943 12295 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:55.012399 11698 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 2 LEARNER]: Refusing update from remote peer b891814873234c21a8e8b4586c05d90a: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260809 07:55:55.013159 12295 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:55:55.017868 12275 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, ecc70316b54f4e588252a8dea3c7fceb (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.018471 11696 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, ecc70316b54f4e588252a8dea3c7fceb (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.020043 11189 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, ecc70316b54f4e588252a8dea3c7fceb (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.023042 12275 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.023304 12275 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:55.023890 12288 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.024098 12288 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc70316b54f4e588252a8dea3c7fceb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:55:55.024880 12280 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.025094 12280 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:55.024410 12003 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, ecc70316b54f4e588252a8dea3c7fceb (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.025276 12306 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.025563 12306 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:55.025727 12275 sys_catalog.cc:455] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.025946 12275 sys_catalog.cc:458] T 00000000000000000000000000000000 P b891814873234c21a8e8b4586c05d90a [sys.catalog]: This master's current role is: LEADER
I20260809 07:55:55.027542 12283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.027848 12283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c39c13c93ce4cd9b640d6b18c3ed10d [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:55:55.030144 12310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:55.030400 12300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b891814873234c21a8e8b4586c05d90a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2ea42ac0f96a4be6a2389586082f3fc1" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 39721 } } peers { permanent_uuid: "b891814873234c21a8e8b4586c05d90a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38377 } } peers { permanent_uuid: "4c39c13c93ce4cd9b640d6b18c3ed10d" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 37401 } } peers { permanent_uuid: "ecc70316b54f4e588252a8dea3c7fceb" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 37057 } attrs { promote: false } } }
I20260809 07:55:55.030618 12300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea42ac0f96a4be6a2389586082f3fc1 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:55:55.030807 12308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:55.032037 12313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:55.032233 12286 server_base.cc:1061] running on GCE node
I20260809 07:55:55.032928 12286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:55.034139 12286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:55.035358 12286 hybrid_clock.cc:648] HybridClock initialized: now 1786262155035177 us; error 135 us; skew 500 ppm
I20260809 07:55:55.035674 12286 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:55.037860 12286 webserver.cc:533] Webserver started at http://127.26.37.1:41463/ using document root <none> and password file <none>
I20260809 07:55:55.038621 12286 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:55.038765 12286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:55.044477 12286 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:55:55.047318 12322 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:55.048753 12286 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:55:55.048978 12286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f7724a564c8240378a695d2d0c5029d6"
format_stamp: "Formatted at 2026-08-09 07:55:51 on dist-test-slave-bw88"
I20260809 07:55:55.050105 12286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:55.074599 12286 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:55.075506 12286 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:55.075798 12286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:55.076752 12286 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:55.078516 12286 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:55.078657 12286 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:55.078792 12286 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:55.078884 12286 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:55:55.079312 12286 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:55:55.079423 12286 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:55:55.079489 12286 fault_injection.cc:59] injecting fault for kudu (pid 12286): FLAGS_fault_before_start (process will exit)
I20260809 07:55:55.104517 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39287
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401
--webserver_port=41463
--webserver_interface=127.26.37.1
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057 with env {}
W20260809 07:55:55.465430 12332 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:55.465749 12332 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:55.465889 12332 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:55.477996 12332 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:55.478217 12332 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:55:55.488188 12332 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42079,127.26.37.21:40931,127.26.37.22:44611,127.26.37.23:46433,127.26.37.24:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39287
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41463
--tserver_master_addrs=127.26.37.62:39721,127.26.37.61:38377,127.26.37.60:37401,127.26.37.59:37057
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:55.490036 12332 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:55.491838 12332 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:55.501411 12341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:55.501435 12342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:55.503189 12344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:55.503536 12332 server_base.cc:1061] running on GCE node
I20260809 07:55:55.504199 12332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:55:55.505421 12332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:55:55.506669 12332 hybrid_clock.cc:648] HybridClock initialized: now 1786262155506510 us; error 124 us; skew 500 ppm
I20260809 07:55:55.507045 12332 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:55:55.509207 12332 webserver.cc:533] Webserver started at http://127.26.37.1:41463/ using document root <none> and password file <none>
I20260809 07:55:55.510068 12332 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:55.510221 12332 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:55.515790 12332 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:55:55.518608 12353 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:55.519934 12332 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:55:55.520139 12332 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "f7724a564c8240378a695d2d0c5029d6"
format_stamp: "Formatted at 2026-08-09 07:55:51 on dist-test-slave-bw88"
I20260809 07:55:55.520908 12332 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:55.542565 12332 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:55.543480 12332 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:55.543829 12332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:55.544776 12332 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:55:55.546636 12332 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:55:55.546720 12332 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:55.546829 12332 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:55:55.546882 12332 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:55.587172 12332 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39287
I20260809 07:55:55.587211 12513 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39287 every 8 connection(s)
I20260809 07:55:55.588905 12332 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:55:55.591236 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12332
I20260809 07:55:55.591516 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12332
I20260809 07:55:55.628206 12515 heartbeater.cc:344] Connected to a master server at 127.26.37.60:37401
I20260809 07:55:55.628471 12507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:55.628641 12515 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:55.714969 12517 heartbeater.cc:344] Connected to a master server at 127.26.37.62:39721
I20260809 07:55:55.716651 12517 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:55.716941 12514 heartbeater.cc:344] Connected to a master server at 127.26.37.59:37057
I20260809 07:55:55.717108 12514 heartbeater.cc:461] Registering TS with master...
I20260809 07:55:55.717398 12515 heartbeater.cc:507] Master 127.26.37.60:37401 requested a full tablet report, sending...
I20260809 07:55:55.717440 12517 heartbeater.cc:507] Master 127.26.37.62:39721 requested a full tablet report, sending...
I20260809 07:55:55.717728 12514 heartbeater.cc:507] Master 127.26.37.59:37057 requested a full tablet report, sending...
I20260809 07:55:55.719028 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12332
I20260809 07:55:55.719553 11682 ts_manager.cc:194] Re-registered known tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
I20260809 07:55:55.719815 11989 ts_manager.cc:194] Re-registered known tserver with Master: f7724a564c8240378a695d2d0c5029d6 (127.26.37.1:39287)
W20260809 07:55:55.744016 11967 connection.cc:573] server connection from 127.26.37.1:55847 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:55:55.744289 11664 connection.cc:573] server connection from 127.26.37.1:49835 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:55:55.744374 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11941
W20260809 07:55:55.746078 12533 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:55:55.615348 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:35367 (local address 127.26.37.61:38377)
0809 07:55:55.629315 (+ 13967us) server_negotiation.cc:207] Beginning negotiation
0809 07:55:55.629325 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:55:55.629364 (+ 39us) server_negotiation.cc:408] Connection header received
0809 07:55:55.629441 (+ 77us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:55:55.629450 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:55:55.629548 (+ 98us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:55:55.629693 (+ 145us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:55:55.716137 (+ 86444us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:55:55.717917 (+ 1780us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:55:55.744987 (+ 27070us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:35367: BlockingRecv error: recv got EOF from 127.26.37.1:35367 (error 108)
Metrics: {"server-negotiator.queue_time_us":13823,"thread_start_us":13703,"threads_started":1}
I20260809 07:55:55.750633 12033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:55.854727 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11941
W20260809 07:55:55.880151 11158 connection.cc:573] client connection to 127.26.37.62:39721 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:55:55.880611 11134 master_runner.cc:187] Error getting master registration for 127.26.37.62:39721: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260809 07:55:55.880800 11134 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:55:55.880919 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11038
I20260809 07:55:55.882079 11121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:55.991183 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11038
I20260809 07:55:56.024014 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11134
I20260809 07:55:56.025259 11212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:56.138458 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11134
I20260809 07:55:56.164783 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 11485
I20260809 07:55:56.166153 11724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:55:56.278609 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 11485
Shutting down hive metastore.
2026-08-09T07:55:56Z chronyd exiting
2026-08-09T07:55:56Z chronyd exiting
2026-08-09T07:55:56Z chronyd exiting
2026-08-09T07:55:56Z chronyd exiting
2026-08-09T07:55:56Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/5 (23574 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/6
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:55:56 dist-test-slave-bw88 krb5kdc[12555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:55:56 dist-test-slave-bw88 krb5kdc[12555](info): set up 2 sockets
Aug 09 07:55:56 dist-test-slave-bw88 krb5kdc[12555](info): commencing operation
krb5kdc: starting...
W20260809 07:55:58.962255 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.199s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:55:59 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:55:59.326665 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38983
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:55:59.684456 12572 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:55:59.684813 12572 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:55:59.684986 12572 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:55:59.693296 12572 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:55:59.693408 12572 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:55:59.693511 12572 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:55:59.693593 12572 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:55:59.704330 12572 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:55:59.706113 12572 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:55:59.707652 12572 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:55:59.716339 12581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:59.716296 12585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:55:59.717150 12587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:55:59.717818 12572 server_base.cc:1061] running on GCE node
I20260809 07:55:59.718608 12572 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:55:59.718813 12572 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:55:59.718930 12572 hybrid_clock.cc:648] HybridClock initialized: now 1786262159718885 us; error 0 us; skew 500 ppm
I20260809 07:55:59.719292 12572 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:55:59 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:55:59.722821 12572 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:55:59.724620 12572 webserver.cc:533] Webserver started at http://127.26.37.62:40693/ using document root <none> and password file <none>
I20260809 07:55:59.725291 12572 fs_manager.cc:362] Metadata directory not provided
I20260809 07:55:59.725508 12572 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:55:59.725889 12572 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:55:59.728282 12572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "acd293196bbe49588c9b50b8c530b5e2"
format_stamp: "Formatted at 2026-08-09 07:55:59 on dist-test-slave-bw88"
I20260809 07:55:59.729046 12572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "acd293196bbe49588c9b50b8c530b5e2"
format_stamp: "Formatted at 2026-08-09 07:55:59 on dist-test-slave-bw88"
I20260809 07:55:59.734411 12572 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 07:55:59.737593 12596 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:55:59.738848 12572 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:55:59.739027 12572 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "acd293196bbe49588c9b50b8c530b5e2"
format_stamp: "Formatted at 2026-08-09 07:55:59 on dist-test-slave-bw88"
I20260809 07:55:59.739207 12572 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:55:59.754766 12572 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:55:59.762385 12572 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:55:59.762707 12572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:55:59.784603 12572 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:38983
I20260809 07:55:59.784627 12671 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:38983 every 8 connection(s)
I20260809 07:55:59.786708 12572 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:55:59.789878 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12572
I20260809 07:55:59.790097 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:55:59.791584 12674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:55:59.800282 12674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:55:59 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:55:59.824115 12674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:55:59.825430 12599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37293: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
W20260809 07:55:59.827075 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37293: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37293: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:55:59.876374 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } attempt: 1
W20260809 07:55:59.878477 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37293: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37293: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:55:59.926789 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37293
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:55:59.946756 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } attempt: 2
W20260809 07:55:59.948874 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37293: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37293: connect: Connection refused (error 111)
I20260809 07:56:00.040194 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } attempt: 3
W20260809 07:56:00.042438 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37293: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37293: connect: Connection refused (error 111)
I20260809 07:56:00.185704 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } attempt: 4
W20260809 07:56:00.188081 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37293: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37293: connect: Connection refused (error 111)
W20260809 07:56:00.272818 12688 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:00.273147 12688 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:00.273233 12688 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:00.281739 12688 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:00.281842 12688 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:00.281899 12688 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:00.281948 12688 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:00.292578 12688 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:00.294278 12688 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:00.295603 12688 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:00.303771 12695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:00.303987 12694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:00.304600 12698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:00.305161 12688 server_base.cc:1061] running on GCE node
I20260809 07:56:00.305989 12688 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:00.306174 12688 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:00.306303 12688 hybrid_clock.cc:648] HybridClock initialized: now 1786262160306241 us; error 0 us; skew 500 ppm
I20260809 07:56:00.306645 12688 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:00 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:00.310024 12688 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:56:00.311995 12688 webserver.cc:533] Webserver started at http://127.26.37.61:35903/ using document root <none> and password file <none>
I20260809 07:56:00.312721 12688 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:00.312872 12688 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:00.313235 12688 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:00.315698 12688 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "7b2264025dd4458f8aeca83cd1db2ead"
format_stamp: "Formatted at 2026-08-09 07:56:00 on dist-test-slave-bw88"
I20260809 07:56:00.316448 12688 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "7b2264025dd4458f8aeca83cd1db2ead"
format_stamp: "Formatted at 2026-08-09 07:56:00 on dist-test-slave-bw88"
I20260809 07:56:00.321746 12688 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 07:56:00.324934 12704 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:00.326318 12688 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:00.326493 12688 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "7b2264025dd4458f8aeca83cd1db2ead"
format_stamp: "Formatted at 2026-08-09 07:56:00 on dist-test-slave-bw88"
I20260809 07:56:00.326681 12688 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:00.345188 12688 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:00.348845 12688 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:00.349128 12688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:00.370007 12688 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:37293
I20260809 07:56:00.370045 12780 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:37293 every 8 connection(s)
I20260809 07:56:00.371476 12688 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:56:00.375577 12782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:00.376458 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12688
I20260809 07:56:00.376669 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:56:00.384346 12782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:00 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:00.407699 12782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
Aug 09 07:56:00 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:00.426168 12782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:00.427273 12706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
W20260809 07:56:00.428984 12782 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:00.478384 12782 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 1
W20260809 07:56:00.486347 12782 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
I20260809 07:56:00.487332 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } attempt: 5
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
Aug 09 07:56:00 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:00.504884 12674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:00.506769 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:00.532294 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:46179
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:00.554628 12782 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 2
W20260809 07:56:00.556779 12782 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
I20260809 07:56:00.557998 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 1
W20260809 07:56:00.559960 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
I20260809 07:56:00.628286 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 2
W20260809 07:56:00.630445 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
I20260809 07:56:00.648020 12782 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 3
W20260809 07:56:00.650156 12782 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
I20260809 07:56:00.736711 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 3
W20260809 07:56:00.739167 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
I20260809 07:56:00.793426 12782 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 4
W20260809 07:56:00.795620 12782 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
W20260809 07:56:00.889596 12796 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:00.889969 12796 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:00.890148 12796 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:00.899014 12796 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:00.899164 12796 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:00.899262 12796 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:00.899362 12796 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:00.910020 12796 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:46179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:00.911617 12796 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:00.913038 12796 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260809 07:56:00.916427 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 4
W20260809 07:56:00.918507 12674 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46179: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46179: connect: Connection refused (error 111)
W20260809 07:56:00.922050 12804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:00.922128 12810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:00.922080 12805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:00.923262 12796 server_base.cc:1061] running on GCE node
I20260809 07:56:00.923979 12796 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:00.924170 12796 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:00.924309 12796 hybrid_clock.cc:648] HybridClock initialized: now 1786262160924270 us; error 0 us; skew 500 ppm
I20260809 07:56:00.924624 12796 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:00 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:00.928155 12796 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:56:00.930011 12796 webserver.cc:533] Webserver started at http://127.26.37.60:35399/ using document root <none> and password file <none>
I20260809 07:56:00.930681 12796 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:00.930781 12796 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:00.931067 12796 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:00.933297 12796 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "4769f97c257149ef9466266887e582e0"
format_stamp: "Formatted at 2026-08-09 07:56:00 on dist-test-slave-bw88"
I20260809 07:56:00.934073 12796 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "4769f97c257149ef9466266887e582e0"
format_stamp: "Formatted at 2026-08-09 07:56:00 on dist-test-slave-bw88"
I20260809 07:56:00.939244 12796 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:56:00.942443 12819 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:00.943871 12796 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:56:00.944103 12796 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "4769f97c257149ef9466266887e582e0"
format_stamp: "Formatted at 2026-08-09 07:56:00 on dist-test-slave-bw88"
I20260809 07:56:00.944314 12796 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:00.972302 12796 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:00.976828 12796 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:00.977196 12796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:00.999980 12796 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:46179
I20260809 07:56:01.000000 12899 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:46179 every 8 connection(s)
I20260809 07:56:01.001873 12796 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:56:01.006243 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:01.007875 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 12796
I20260809 07:56:01.008087 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:56:01.014873 12900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:01 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:01 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:01 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:01.043033 12900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:01 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:01 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:01.059773 12900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:01 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:01.078748 12900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: Bootstrap starting.
I20260809 07:56:01.082151 12900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:01.083307 12900 log.cc:826] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:01.086572 12900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: No bootstrap required, opened a new log
I20260809 07:56:01.094255 12900 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
I20260809 07:56:01.094614 12900 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:01.094753 12900 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4769f97c257149ef9466266887e582e0, State: Initialized, Role: FOLLOWER
I20260809 07:56:01.094847 12782 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 5
I20260809 07:56:01.095518 12900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
I20260809 07:56:01.096589 12917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.096971 12917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:01.097776 12900 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 07:56:01 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
W20260809 07:56:01.113871 12930 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:01.113991 12930 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:01.116714 12782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: Bootstrap starting.
I20260809 07:56:01.120744 12782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:01.110893 12796 master_runner.cc:187] Error getting master registration for 127.26.37.62:38983: OK, instance_id { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" instance_seqno: 1786262159778231 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:56:01.122148 12782 log.cc:826] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:01.122421 12796 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:56:01.124624 12782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: No bootstrap required, opened a new log
I20260809 07:56:01.130376 12782 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
I20260809 07:56:01.130615 12782 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:01.130797 12782 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b2264025dd4458f8aeca83cd1db2ead, State: Initialized, Role: FOLLOWER
I20260809 07:56:01.131461 12782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
I20260809 07:56:01.132455 12937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.132812 12937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:01.133564 12782 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:56:01.144466 12954 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:01.144711 12954 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:01.141916 12688 master_runner.cc:187] Error getting master registration for 127.26.37.62:38983: OK, instance_id { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" instance_seqno: 1786262159778231 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:56:01.148640 12688 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:56:01.194748 12917 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:56:01.195060 12917 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
I20260809 07:56:01.195751 12674 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } attempt: 5
I20260809 07:56:01.197086 12917 leader_election.cc:290] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acd293196bbe49588c9b50b8c530b5e2 (127.26.37.62:38983), 7b2264025dd4458f8aeca83cd1db2ead (127.26.37.61:37293)
I20260809 07:56:01.199020 12743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4769f97c257149ef9466266887e582e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b2264025dd4458f8aeca83cd1db2ead" is_pre_election: true
I20260809 07:56:01.199627 12743 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4769f97c257149ef9466266887e582e0 in term 0.
I20260809 07:56:01.200630 12822 leader_election.cc:304] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4769f97c257149ef9466266887e582e0, 7b2264025dd4458f8aeca83cd1db2ead; no voters:
I20260809 07:56:01.201215 12917 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:56:01.201398 12917 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:56:01.201511 12917 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:01.205015 12917 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
Aug 09 07:56:01 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:01.211010 12743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4769f97c257149ef9466266887e582e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b2264025dd4458f8aeca83cd1db2ead"
I20260809 07:56:01.211215 12743 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:01.214701 12743 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4769f97c257149ef9466266887e582e0 in term 1.
I20260809 07:56:01.215191 12822 leader_election.cc:304] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4769f97c257149ef9466266887e582e0, 7b2264025dd4458f8aeca83cd1db2ead; no voters:
I20260809 07:56:01.215970 12957 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:56:01.222419 12674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: Bootstrap starting.
I20260809 07:56:01.226070 12917 leader_election.cc:290] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [CANDIDATE]: Term 1 election: Requested vote from peers acd293196bbe49588c9b50b8c530b5e2 (127.26.37.62:38983), 7b2264025dd4458f8aeca83cd1db2ead (127.26.37.61:37293)
W20260809 07:56:01.229521 12830 tablet.cc:2406] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:56:01.235276 12674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:01.238111 12674 log.cc:826] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:01.240700 12957 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 1 LEADER]: Becoming Leader. State: Replica: 4769f97c257149ef9466266887e582e0, State: Running, Role: LEADER
I20260809 07:56:01.241607 12957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
I20260809 07:56:01.251042 12674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: No bootstrap required, opened a new log
I20260809 07:56:01.256645 12917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4769f97c257149ef9466266887e582e0. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.256881 12917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:01.264701 12674 raft_consensus.cc:359] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
I20260809 07:56:01.265393 12674 raft_consensus.cc:385] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:01.265723 12674 raft_consensus.cc:740] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd293196bbe49588c9b50b8c530b5e2, State: Initialized, Role: FOLLOWER
I20260809 07:56:01.267098 12674 consensus_queue.cc:260] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } }
I20260809 07:56:01.270011 12964 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:56:01.270213 12965 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.270539 12965 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:01.272351 12964 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:56:01.273419 12639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4769f97c257149ef9466266887e582e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acd293196bbe49588c9b50b8c530b5e2" is_pre_election: true
I20260809 07:56:01.273830 12639 raft_consensus.cc:2468] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4769f97c257149ef9466266887e582e0 in term 0.
I20260809 07:56:01.273137 12638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4769f97c257149ef9466266887e582e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acd293196bbe49588c9b50b8c530b5e2"
I20260809 07:56:01.274887 12638 raft_consensus.cc:3060] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:01.278255 12638 raft_consensus.cc:2468] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4769f97c257149ef9466266887e582e0 in term 1.
I20260809 07:56:01.280198 12674 sys_catalog.cc:565] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:01.292310 12743 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 1 FOLLOWER]: Refusing update from remote peer 4769f97c257149ef9466266887e582e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:56:01.293696 12917 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:56:01.300768 12638 raft_consensus.cc:1275] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 1 FOLLOWER]: Refusing update from remote peer 4769f97c257149ef9466266887e582e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:56:01.301783 12917 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:56:01.339265 12965 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4769f97c257149ef9466266887e582e0. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.339552 12965 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:01.341138 12967 mvcc.cc:204] Tried to move back new op lower bound from 7316529812637708288 to 7316529812455956480. Current Snapshot: MvccSnapshot[applied={T|T < 7316529812637708288}]
I20260809 07:56:01.345188 12975 mvcc.cc:204] Tried to move back new op lower bound from 7316529812637708288 to 7316529812455956480. Current Snapshot: MvccSnapshot[applied={T|T < 7316529812637708288}]
W20260809 07:56:01.351791 12712 tablet.cc:2406] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:56:01.353546 12957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.353842 12957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:01.355139 12965 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.355333 12965 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:01.356155 12964 catalog_manager.cc:1383] Generated new cluster ID: d431bfe1d7d74a97aebe5c47d6f87ca8
I20260809 07:56:01.356281 12964 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:56:01.359081 12937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4769f97c257149ef9466266887e582e0. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.359347 12937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:01.359683 12917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.359927 12917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:01.369923 12937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } }
I20260809 07:56:01.370146 12937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:01.381445 12996 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:56:01.385744 12964 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:56:01.387554 12964 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:56:01.402185 12964 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: Generated new TSK 0
I20260809 07:56:01.403260 12964 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:01.597849 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:01.975574 13004 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:01.975849 13004 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:01.975932 13004 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:01.984174 13004 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:01.984349 13004 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:01.994578 13004 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:01.996271 13004 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:01.997599 13004 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:02.008040 13010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:02.008749 13011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:02.009049 13013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:02.009869 13004 server_base.cc:1061] running on GCE node
I20260809 07:56:02.010531 13004 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:02.010725 13004 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:02.010800 13004 hybrid_clock.cc:648] HybridClock initialized: now 1786262162010781 us; error 0 us; skew 500 ppm
I20260809 07:56:02.011133 13004 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:02.014976 13004 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:02.016695 13004 webserver.cc:533] Webserver started at http://127.26.37.1:34021/ using document root <none> and password file <none>
I20260809 07:56:02.017511 13004 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:02.017654 13004 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:02.018023 13004 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:02.020687 13004 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "b00c261880e7466ca9cb7dc72072b24d"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:02.021545 13004 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "b00c261880e7466ca9cb7dc72072b24d"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:02.026710 13004 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 07:56:02.030025 13019 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:02.031235 13004 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:56:02.031404 13004 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "b00c261880e7466ca9cb7dc72072b24d"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:02.031584 13004 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:02.053812 13004 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:02.057273 13004 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:02.057590 13004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:02.058447 13004 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:02.060063 13004 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:02.060163 13004 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:02.060310 13004 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:02.060380 13004 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:02.095309 13004 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:33207
I20260809 07:56:02.095395 13191 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:33207 every 8 connection(s)
I20260809 07:56:02.096830 13004 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:02.100696 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13004
I20260809 07:56:02.100908 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:02.135322 13196 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38983
I20260809 07:56:02.135759 13196 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:02.136785 13197 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37293
I20260809 07:56:02.137012 13197 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:02.137437 13196 heartbeater.cc:507] Master 127.26.37.62:38983 requested a full tablet report, sending...
I20260809 07:56:02.137691 13197 heartbeater.cc:507] Master 127.26.37.61:37293 requested a full tablet report, sending...
I20260809 07:56:02.140174 12623 ts_manager.cc:194] Registered new tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:02.140172 12728 ts_manager.cc:194] Registered new tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:02.142257 13195 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46179
I20260809 07:56:02.142402 13195 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:02.142895 13195 heartbeater.cc:507] Master 127.26.37.60:46179 requested a full tablet report, sending...
I20260809 07:56:02.144608 12843 ts_manager.cc:194] Registered new tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:02.146795 12843 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:53703
I20260809 07:56:02.146991 12954 catalog_manager.cc:1295] Loaded cluster ID: d431bfe1d7d74a97aebe5c47d6f87ca8
I20260809 07:56:02.147073 12954 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: loading cluster ID for follower catalog manager: success
I20260809 07:56:02.150390 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:02.150662 12954 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: acquiring CA information for follower catalog manager: success
I20260809 07:56:02.152853 12954 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for kudu/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:02.289122 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:43745
--webserver_interface=127.26.37.59
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:02.386503 12996 catalog_manager.cc:1295] Loaded cluster ID: d431bfe1d7d74a97aebe5c47d6f87ca8
I20260809 07:56:02.386643 12996 catalog_manager.cc:1588] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: loading cluster ID for follower catalog manager: success
I20260809 07:56:02.390620 12996 catalog_manager.cc:1610] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: acquiring CA information for follower catalog manager: success
I20260809 07:56:02.393074 12996 catalog_manager.cc:1638] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:56:02.654292 13217 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:02.654685 13217 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:02.654831 13217 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:02.663492 13217 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:02.663641 13217 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:02.663748 13217 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:02.663822 13217 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:02.674391 13217 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:43745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:02.676126 13217 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:02.677562 13217 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:02.686352 13225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:02.686323 13226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:02.687960 13228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:02.688427 13217 server_base.cc:1061] running on GCE node
I20260809 07:56:02.689464 13217 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:02.689690 13217 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:02.689802 13217 hybrid_clock.cc:648] HybridClock initialized: now 1786262162689778 us; error 0 us; skew 500 ppm
I20260809 07:56:02.690269 13217 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:02.694794 13217 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:56:02.696992 13217 webserver.cc:533] Webserver started at http://127.26.37.59:38259/ using document root <none> and password file <none>
I20260809 07:56:02.697877 13217 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:02.698019 13217 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:02.698402 13217 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:02.700953 13217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "03bcf281167041a882b5f0ac147dc746"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:02.701864 13217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "03bcf281167041a882b5f0ac147dc746"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:02.707353 13217 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:56:02.710894 13238 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:02.712342 13217 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:56:02.712499 13217 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "03bcf281167041a882b5f0ac147dc746"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:02.712675 13217 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:02.727453 13217 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:02.731477 13217 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:02.731787 13217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:02.754336 13217 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:43745
I20260809 07:56:02.754355 13309 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:43745 every 8 connection(s)
I20260809 07:56:02.755863 13217 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:56:02.760399 13310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:02.763131 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13217
I20260809 07:56:02.768236 13310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:02.788934 13310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:02.805789 13310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:02.817740 13310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262162, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:02.833583 13310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Bootstrap starting.
I20260809 07:56:02.836997 13310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:02.838188 13310 log.cc:826] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:02.840691 13310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: No bootstrap required, opened a new log
I20260809 07:56:02.845964 13310 raft_consensus.cc:359] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } }
I20260809 07:56:02.846262 13310 raft_consensus.cc:385] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:02.846356 13310 raft_consensus.cc:740] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03bcf281167041a882b5f0ac147dc746, State: Initialized, Role: FOLLOWER
I20260809 07:56:02.846946 13310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } }
I20260809 07:56:02.847803 13323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } } }
I20260809 07:56:02.848096 13323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:02.848805 13310 sys_catalog.cc:565] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:56:02.858029 13337 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:02.858186 13337 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:02.865483 13217 master_runner.cc:429] Detected that this master 03bcf281167041a882b5f0ac147dc746 is joining an existing cluster
I20260809 07:56:02.865603 13217 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:56:02.866518 13217 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:56:02.866856 13217 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:56:02.867051 13217 tablet_replica.cc:333] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: stopping tablet replica
I20260809 07:56:02.868580 13217 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:43745
Aug 09 07:56:02 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262160, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:02.932536 12843 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:43745: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "03bcf281167041a882b5f0ac147dc746"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 43745
}
attrs {
promote: true
}
}
dest_uuid: "4769f97c257149ef9466266887e582e0"
cas_config_opid_index: -1
I20260809 07:56:02.934291 12843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } }
I20260809 07:56:02.937644 12639 raft_consensus.cc:1275] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 1 FOLLOWER]: Refusing update from remote peer 4769f97c257149ef9466266887e582e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260809 07:56:02.938558 12826 consensus_peers.cc:597] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 -> Peer 03bcf281167041a882b5f0ac147dc746 (127.26.37.59:43745): Couldn't send request to peer 03bcf281167041a882b5f0ac147dc746. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:56:02.938802 12960 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:56:02.940637 12740 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 1 FOLLOWER]: Refusing update from remote peer 4769f97c257149ef9466266887e582e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:56:02.941318 13355 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:56:02.941995 13355 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 03bcf281167041a882b5f0ac147dc746 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:02.942703 12639 raft_consensus.cc:2955] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 03bcf281167041a882b5f0ac147dc746 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:02.946170 13355 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:43745
I20260809 07:56:02.946900 13217 master.cc:562] Master@127.26.37.59:43745 shutting down...
I20260809 07:56:02.975648 13360 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:02.976054 13360 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:02.977247 13217 master.cc:584] Master@127.26.37.59:43745 shutdown complete.
I20260809 07:56:02.977396 13217 master_runner.cc:306] Clearing existing system tablet
I20260809 07:56:02.977124 13355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:02.977123 12960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:02.977802 12960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:02.977828 13355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:02.977900 13217 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:02.977996 13217 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:02.980360 13217 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:56:02.980530 12960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:02.980792 12960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:02.981540 12740 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 03bcf281167041a882b5f0ac147dc746 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:02.982810 13365 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:02.983349 13217 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:02.983628 13217 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:56:02.983712 13217 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "03bcf281167041a882b5f0ac147dc746"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:02.986443 13367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:02.986858 13367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:02.988415 13217 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:56:02.993085 13217 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:56:02.993260 13217 log.cc:1199] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:56:02.993775 13217 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Deleting consensus metadata
I20260809 07:56:02.994024 13217 master_runner.cc:316] Copying system tablet from 127.26.37.60:46179
I20260809 07:56:02.996444 13217 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.60:46179
I20260809 07:56:03.009835 12876 tablet_copy_service.cc:140] P 4769f97c257149ef9466266887e582e0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 03bcf281167041a882b5f0ac147dc746 ({username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:39636)
I20260809 07:56:03.010170 12876 tablet_copy_service.cc:161] P 4769f97c257149ef9466266887e582e0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 03bcf281167041a882b5f0ac147dc746 at {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:39636: session id = 03bcf281167041a882b5f0ac147dc746-00000000000000000000000000000000
I20260809 07:56:03.013195 12876 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:56:03.015318 13217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:56:03.020227 13217 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: tablet copy: Starting download of 0 data blocks...
I20260809 07:56:03.020587 13217 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: tablet copy: Starting download of 1 WAL segments...
I20260809 07:56:03.023111 13217 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:56:03.026445 12876 tablet_copy_service.cc:342] P 4769f97c257149ef9466266887e582e0: Request end of tablet copy session 03bcf281167041a882b5f0ac147dc746-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:39636
I20260809 07:56:03.026667 12876 tablet_copy_service.cc:434] P 4769f97c257149ef9466266887e582e0: ending tablet copy session 03bcf281167041a882b5f0ac147dc746-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 03bcf281167041a882b5f0ac147dc746
I20260809 07:56:03.038852 13217 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:03.041558 13376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:03.041656 13375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:03.043018 13378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:03.046554 13217 server_base.cc:1061] running on GCE node
I20260809 07:56:03.046916 13217 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:03.047024 13217 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:03.047116 13217 hybrid_clock.cc:648] HybridClock initialized: now 1786262163047064 us; error 0 us; skew 500 ppm
I20260809 07:56:03.047307 13217 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:03 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:03.049832 13217 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:56:03.051043 13217 webserver.cc:533] Webserver started at http://127.26.37.59:43019/ using document root <none> and password file <none>
I20260809 07:56:03.051421 13217 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:03.051502 13217 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:03.054401 13217 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:56:03.056296 13386 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:03.056759 13217 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 07:56:03.056877 13217 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "03bcf281167041a882b5f0ac147dc746"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:03.057034 13217 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:03.073251 13217 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:03.077289 13217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:03.107935 13217 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:43745
I20260809 07:56:03.108026 13467 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:43745 every 8 connection(s)
I20260809 07:56:03.109066 13217 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:56:03.112543 13468 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:56:03.114887 13468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Bootstrap starting.
I20260809 07:56:03.128119 13468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:56:03.128602 13468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: Bootstrap complete.
I20260809 07:56:03.129588 13468 raft_consensus.cc:359] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } }
I20260809 07:56:03.129776 13468 raft_consensus.cc:740] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 03bcf281167041a882b5f0ac147dc746, State: Initialized, Role: LEARNER
I20260809 07:56:03.130031 13468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } }
I20260809 07:56:03.130851 13471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:03.131196 13468 sys_catalog.cc:565] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:03.131171 13471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:56:03.139870 13486 catalog_manager.cc:1295] Loaded cluster ID: d431bfe1d7d74a97aebe5c47d6f87ca8
I20260809 07:56:03.139991 13486 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: loading cluster ID for follower catalog manager: success
I20260809 07:56:03.143476 13486 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: acquiring CA information for follower catalog manager: success
I20260809 07:56:03.145593 13486 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:56:03.149748 13195 heartbeater.cc:499] Master 127.26.37.60:46179 was elected leader, sending a full tablet report...
I20260809 07:56:03.167659 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13004
Aug 09 07:56:03 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:56:03.186728 13186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:03.275192 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13004
I20260809 07:56:03.300918 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:33207
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--webserver_port=34021
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:03.368463 13427 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:56:03.369412 13471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4769f97c257149ef9466266887e582e0. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:03.369694 13471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: This master's current role is: LEARNER
W20260809 07:56:03.672130 13490 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:03.672436 13490 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:03.672521 13490 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:03.680452 13490 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:03.680627 13490 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:03.691426 13490 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:33207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34021
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:03.692972 13490 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:03.694350 13490 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:03.704401 13500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:03.705145 13499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:03.705415 13503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:03.705967 13490 server_base.cc:1061] running on GCE node
I20260809 07:56:03.706655 13490 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:03.706856 13490 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:03.706965 13490 hybrid_clock.cc:648] HybridClock initialized: now 1786262163706941 us; error 0 us; skew 500 ppm
I20260809 07:56:03.707321 13490 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:03 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:03.710862 13490 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:03.712558 13490 webserver.cc:533] Webserver started at http://127.26.37.1:34021/ using document root <none> and password file <none>
I20260809 07:56:03.713398 13490 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:03.713573 13490 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:03.719249 13490 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 07:56:03.722306 13512 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:03.723729 13490 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:03.723862 13490 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "b00c261880e7466ca9cb7dc72072b24d"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:03.724548 13490 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:03.740863 13490 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:03.744858 13490 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:03.745090 13490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:03.746061 13490 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:03.747532 13490 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:03.747627 13490 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:03.747706 13490 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:03.747761 13490 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:03.784121 13490 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:33207
I20260809 07:56:03.784173 13662 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:33207 every 8 connection(s)
I20260809 07:56:03.785969 13490 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:03.789189 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13490
Aug 09 07:56:03 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:03 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:03 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:56:03 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262163, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:03.842298 13667 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38983
I20260809 07:56:03.842612 13664 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46179
I20260809 07:56:03.842777 13664 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:03.842782 13667 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:03.842890 13665 heartbeater.cc:344] Connected to a master server at 127.26.37.59:43745
I20260809 07:56:03.843036 13665 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:03.843820 13664 heartbeater.cc:507] Master 127.26.37.60:46179 requested a full tablet report, sending...
I20260809 07:56:03.843899 13665 heartbeater.cc:507] Master 127.26.37.59:43745 requested a full tablet report, sending...
I20260809 07:56:03.844238 13667 heartbeater.cc:507] Master 127.26.37.62:38983 requested a full tablet report, sending...
I20260809 07:56:03.846339 12623 ts_manager.cc:194] Re-registered known tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:03.846557 13411 ts_manager.cc:194] Registered new tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:03.847249 13669 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37293
I20260809 07:56:03.847393 13669 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:03.847748 13669 heartbeater.cc:507] Master 127.26.37.61:37293 requested a full tablet report, sending...
I20260809 07:56:03.848380 12843 ts_manager.cc:194] Re-registered known tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:03.848868 12728 ts_manager.cc:194] Re-registered known tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:03.849337 12843 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:45177
I20260809 07:56:03.855597 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:03.863265 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:03.863392 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12572
I20260809 07:56:03.864331 12667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:03.955600 13683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:03.956302 13683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:03.957185 13683 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0: attempting to promote NON_VOTER 03bcf281167041a882b5f0ac147dc746 to VOTER
I20260809 07:56:03.957974 13683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } }
I20260809 07:56:03.960522 13427 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 1 LEARNER]: Refusing update from remote peer 4769f97c257149ef9466266887e582e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:03.961555 12740 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 1 FOLLOWER]: Refusing update from remote peer 4769f97c257149ef9466266887e582e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:03.961447 13685 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:03.962265 13684 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:03.966524 13683 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 03bcf281167041a882b5f0ac147dc746 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:03.967234 13427 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 03bcf281167041a882b5f0ac147dc746 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:03.969576 13683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:03.969825 13683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:03.970042 13683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:03.970263 13683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:03.970237 13688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:03.970496 13688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:03.970744 13692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:03.971014 13692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bcf281167041a882b5f0ac147dc746 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:03.971266 12740 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 03bcf281167041a882b5f0ac147dc746 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:03.974182 13693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:03.974449 13693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2264025dd4458f8aeca83cd1db2ead [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:03.978770 12639 raft_consensus.cc:1275] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 1 FOLLOWER]: Refusing update from remote peer 4769f97c257149ef9466266887e582e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:03.979198 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12572
I20260809 07:56:03.979387 13684 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 07:56:04.009887 12825 connection.cc:573] client connection to 127.26.37.62:38983 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:56:04.010147 12825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:56:04.011130 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38983
--webserver_interface=127.26.37.62
--webserver_port=40693
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:04.012563 12825 consensus_peers.cc:597] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 -> Peer acd293196bbe49588c9b50b8c530b5e2 (127.26.37.62:38983): Couldn't send request to peer acd293196bbe49588c9b50b8c530b5e2. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:38983: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:56:04.375111 13698 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:04.375514 13698 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:04.375658 13698 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:04.384622 13698 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:04.384732 13698 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:04.384788 13698 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:04.384835 13698 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:04.395253 13698 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=40693
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:04.396749 13698 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:04.398090 13698 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:04.406975 13705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:04.407650 13707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:04.407701 13703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:04.408583 13698 server_base.cc:1061] running on GCE node
I20260809 07:56:04.409235 13698 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:04.409457 13698 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:04.409570 13698 hybrid_clock.cc:648] HybridClock initialized: now 1786262164409548 us; error 0 us; skew 500 ppm
I20260809 07:56:04.409880 13698 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:04 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262164, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:04.413281 13698 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:56:04.415220 13698 webserver.cc:533] Webserver started at http://127.26.37.62:40693/ using document root <none> and password file <none>
I20260809 07:56:04.415902 13698 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:04.416059 13698 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:04.421326 13698 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:04.424091 13714 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:04.425505 13698 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:56:04.425693 13698 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "acd293196bbe49588c9b50b8c530b5e2"
format_stamp: "Formatted at 2026-08-09 07:55:59 on dist-test-slave-bw88"
I20260809 07:56:04.426411 13698 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:04.445655 13698 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:04.449283 13698 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:04.449592 13698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:04.470428 13698 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:38983
I20260809 07:56:04.470512 13783 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:38983 every 8 connection(s)
I20260809 07:56:04.472183 13698 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:56:04.476106 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13698
I20260809 07:56:04.476411 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13490
I20260809 07:56:04.479966 13784 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:56:04.484352 13784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: Bootstrap starting.
I20260809 07:56:04.494246 13657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:04.507113 13784 log.cc:826] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:04.520917 13784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:56:04.521776 13784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: Bootstrap complete.
I20260809 07:56:04.532939 13784 raft_consensus.cc:359] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } }
I20260809 07:56:04.534065 13784 raft_consensus.cc:740] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd293196bbe49588c9b50b8c530b5e2, State: Initialized, Role: FOLLOWER
I20260809 07:56:04.535089 13784 consensus_queue.cc:260] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } }
I20260809 07:56:04.536036 13792 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: true } } }
I20260809 07:56:04.536324 13792 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:04.537563 13784 sys_catalog.cc:565] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:04.549153 13810 catalog_manager.cc:1295] Loaded cluster ID: d431bfe1d7d74a97aebe5c47d6f87ca8
I20260809 07:56:04.549351 13810 catalog_manager.cc:1588] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: loading cluster ID for follower catalog manager: success
I20260809 07:56:04.553694 13810 catalog_manager.cc:1610] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: acquiring CA information for follower catalog manager: success
I20260809 07:56:04.556321 13810 catalog_manager.cc:1638] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:56:04.619001 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13490
I20260809 07:56:04.645521 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:33207
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--webserver_port=34021
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:04.818483 13751 raft_consensus.cc:2955] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 03bcf281167041a882b5f0ac147dc746 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:04.828603 13792 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4769f97c257149ef9466266887e582e0. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:04.828835 13792 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:04.827363 13825 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:04.829370 13825 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd293196bbe49588c9b50b8c530b5e2 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:56:05.024072 13820 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:05.024439 13820 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:05.024612 13820 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:05.033077 13820 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:05.033308 13820 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:05.044129 13820 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:33207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34021
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:05.045852 13820 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:05.047228 13820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:05.057886 13834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:05.057937 13832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:05.058238 13836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:05.059036 13820 server_base.cc:1061] running on GCE node
I20260809 07:56:05.059578 13820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:05.059795 13820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:05.059894 13820 hybrid_clock.cc:648] HybridClock initialized: now 1786262165059872 us; error 0 us; skew 500 ppm
I20260809 07:56:05.060210 13820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:05 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262165, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:05.063860 13820 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:05.065586 13820 webserver.cc:533] Webserver started at http://127.26.37.1:34021/ using document root <none> and password file <none>
I20260809 07:56:05.066300 13820 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:05.066444 13820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:05.071729 13820 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.002s
I20260809 07:56:05.074499 13847 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:05.075794 13820 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:05.075978 13820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "b00c261880e7466ca9cb7dc72072b24d"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:05.076709 13820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:05.099149 13820 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:05.107471 13820 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:05.107764 13820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:05.108814 13820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:05.110503 13820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:05.110642 13820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:05.110762 13820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:05.110852 13820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:05.148067 13820 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:33207
I20260809 07:56:05.148159 14006 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:33207 every 8 connection(s)
I20260809 07:56:05.149899 13820 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:05.154039 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 13820
Aug 09 07:56:05 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262165, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:05 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262165, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:05 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262165, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:05 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262165, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:05.204973 14008 heartbeater.cc:344] Connected to a master server at 127.26.37.59:43745
I20260809 07:56:05.205004 14009 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38983
I20260809 07:56:05.205168 14010 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37293
I20260809 07:56:05.205058 14007 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46179
I20260809 07:56:05.205543 14007 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:05.205531 14008 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:05.205533 14009 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:05.205849 14010 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:05.206875 14009 heartbeater.cc:507] Master 127.26.37.62:38983 requested a full tablet report, sending...
I20260809 07:56:05.206943 14008 heartbeater.cc:507] Master 127.26.37.59:43745 requested a full tablet report, sending...
I20260809 07:56:05.207489 14010 heartbeater.cc:507] Master 127.26.37.61:37293 requested a full tablet report, sending...
I20260809 07:56:05.207863 14007 heartbeater.cc:507] Master 127.26.37.60:46179 requested a full tablet report, sending...
I20260809 07:56:05.209022 12843 ts_manager.cc:194] Re-registered known tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:05.209620 13738 ts_manager.cc:194] Registered new tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:05.210178 12843 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:35121
I20260809 07:56:05.211648 12728 ts_manager.cc:194] Re-registered known tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:05.211786 13411 ts_manager.cc:194] Re-registered known tserver with Master: b00c261880e7466ca9cb7dc72072b24d (127.26.37.1:33207)
I20260809 07:56:05.233851 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13820
W20260809 07:56:05.235169 14025 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:05.231473 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.26.37.1:33207)
0809 07:56:05.232286 (+ 813us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:05.232300 (+ 14us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:05.232332 (+ 32us) server_negotiation.cc:408] Connection header received
0809 07:56:05.232459 (+ 127us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:05.232466 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:05.232667 (+ 201us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:05.232824 (+ 157us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:05.233683 (+ 859us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33838: BlockingRecv error: recv got EOF from 127.0.0.1:33838 (error 108)
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":411,"threads_started":1}
W20260809 07:56:05.236016 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13820: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:33207: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:05.236161 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13820
I20260809 07:56:05.263223 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:33207
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--webserver_port=34021
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:05.384012 13684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4769f97c257149ef9466266887e582e0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acd293196bbe49588c9b50b8c530b5e2" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38983 } } peers { permanent_uuid: "7b2264025dd4458f8aeca83cd1db2ead" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37293 } } peers { permanent_uuid: "4769f97c257149ef9466266887e582e0" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46179 } } peers { permanent_uuid: "03bcf281167041a882b5f0ac147dc746" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 43745 } attrs { promote: false } } }
I20260809 07:56:05.384358 13684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4769f97c257149ef9466266887e582e0 [sys.catalog]: This master's current role is: LEADER
W20260809 07:56:05.620296 14029 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:05.620631 14029 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:05.620815 14029 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:56:05.620886 14029 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:05.629105 14029 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:05.629401 14029 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:05.640200 14029 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:33207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34021
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:05.641932 14029 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:05.643306 14029 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:05.653223 14043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:05.653220 14042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:05.654665 14046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:05.655119 14029 server_base.cc:1061] running on GCE node
I20260809 07:56:05.655748 14029 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:05.655879 14029 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:05.655961 14029 hybrid_clock.cc:648] HybridClock initialized: now 1786262165655930 us; error 0 us; skew 500 ppm
I20260809 07:56:05.656200 14029 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:05 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262165, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:05.659679 14029 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:05.661324 14029 webserver.cc:533] Webserver started at http://127.26.37.1:34021/ using document root <none> and password file <none>
I20260809 07:56:05.662075 14029 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:05.662158 14029 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:05.667693 14029 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:56:05.670315 14056 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:05.671654 14029 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:05.671814 14029 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "b00c261880e7466ca9cb7dc72072b24d"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:05.672510 14029 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:05.689143 14029 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:05.692986 14029 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:05.693264 14029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:05.694167 14029 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:05.695641 14029 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:05.695709 14029 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:05.695891 14029 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:05.695945 14029 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:56:05.696367 14029 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:56:05.696431 14029 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:56:05.696537 14029 fault_injection.cc:59] injecting fault for kudu (pid 14029): FLAGS_fault_before_start (process will exit)
I20260809 07:56:05.727594 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:33207
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179
--webserver_port=34021
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:06.084553 14071 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:06.085017 14071 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:06.085177 14071 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:06.093245 14071 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:06.093488 14071 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:06.104122 14071 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:33207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=34021
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38983,127.26.37.61:37293,127.26.37.60:46179,127.26.37.59:43745
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:06.105759 14071 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:06.107187 14071 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:06.117419 14076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:06.117450 14077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:06.119213 14079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:06.119333 14071 server_base.cc:1061] running on GCE node
I20260809 07:56:06.119982 14071 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:06.120170 14071 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:06.120273 14071 hybrid_clock.cc:648] HybridClock initialized: now 1786262166120233 us; error 0 us; skew 500 ppm
I20260809 07:56:06.120584 14071 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:06 dist-test-slave-bw88 krb5kdc[12555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:06.124161 14071 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:06.126147 14071 webserver.cc:533] Webserver started at http://127.26.37.1:34021/ using document root <none> and password file <none>
I20260809 07:56:06.126933 14071 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:06.127075 14071 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:06.133287 14071 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:56:06.136382 14086 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:06.137881 14071 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:06.138038 14071 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "b00c261880e7466ca9cb7dc72072b24d"
format_stamp: "Formatted at 2026-08-09 07:56:02 on dist-test-slave-bw88"
I20260809 07:56:06.138814 14071 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:06.158643 14071 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:06.163017 14071 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:06.163291 14071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:06.164211 14071 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:06.165817 14071 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:06.165910 14071 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:06.166066 14071 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:06.166157 14071 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:06.201745 14071 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:33207
I20260809 07:56:06.201838 14241 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:33207 every 8 connection(s)
I20260809 07:56:06.203296 14071 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:06.212746 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14071
I20260809 07:56:06.213191 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14071
Aug 09 07:56:06 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:06 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:06 dist-test-slave-bw88 krb5kdc[12555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262166, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
W20260809 07:56:06.266445 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14071: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:33207: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:06.266626 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14071
W20260809 07:56:06.296651 14260 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:06.243259 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:53159 (local address 127.26.37.61:37293)
0809 07:56:06.262563 (+ 19304us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:06.262570 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:06.262588 (+ 18us) server_negotiation.cc:408] Connection header received
0809 07:56:06.262655 (+ 67us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:06.262661 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:06.262739 (+ 78us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:06.262879 (+ 140us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:06.265472 (+ 2593us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:06.277106 (+ 11634us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:56:06.295556 (+ 18450us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:53159: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":19159,"thread_start_us":19022,"threads_started":1}
W20260809 07:56:06.296651 14249 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:06.207274 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:36683 (local address 127.26.37.60:46179)
0809 07:56:06.250861 (+ 43587us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:06.250867 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:06.250886 (+ 19us) server_negotiation.cc:408] Connection header received
0809 07:56:06.250974 (+ 88us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:06.250982 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:06.251063 (+ 81us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:06.251210 (+ 147us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:06.252776 (+ 1566us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:06.254693 (+ 1917us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:56:06.257888 (+ 3195us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:06.258971 (+ 1083us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:56:06.262605 (+ 3634us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0809 07:56:06.262644 (+ 39us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0809 07:56:06.262649 (+ 5us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
0809 07:56:06.262688 (+ 39us) server_negotiation.cc:923] calling sasl_server_start()
0809 07:56:06.275691 (+ 13003us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:56:06.295831 (+ 20140us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:36683: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":43465,"thread_start_us":43334,"threads_started":1}
W20260809 07:56:06.296658 14253 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:06.208608 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:51555 (local address 127.26.37.59:43745)
0809 07:56:06.244447 (+ 35839us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:06.244454 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:06.244478 (+ 24us) server_negotiation.cc:408] Connection header received
0809 07:56:06.244557 (+ 79us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:06.244565 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:06.244679 (+ 114us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:06.244830 (+ 151us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:06.253446 (+ 8616us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:06.254931 (+ 1485us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:56:06.257002 (+ 2071us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:06.257515 (+ 513us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:56:06.295650 (+ 38135us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:51555: BlockingRecv error: recv got EOF from 127.26.37.1:51555 (error 108)
Metrics: {"server-negotiator.queue_time_us":35679,"thread_start_us":36559,"threads_started":1}
I20260809 07:56:06.297150 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13698
W20260809 07:56:06.297394 14095 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:06.208009 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:47003 (local address 127.26.37.62:38983)
0809 07:56:06.208221 (+ 212us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:06.208231 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:06.242474 (+ 34243us) server_negotiation.cc:408] Connection header received
0809 07:56:06.242670 (+ 196us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:06.242676 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:06.242766 (+ 90us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:06.242867 (+ 101us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:06.252835 (+ 9968us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:06.254154 (+ 1319us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:56:06.255752 (+ 1598us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:06.256291 (+ 539us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:56:06.261064 (+ 4773us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0809 07:56:06.261100 (+ 36us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0809 07:56:06.261106 (+ 6us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
0809 07:56:06.261146 (+ 40us) server_negotiation.cc:923] calling sasl_server_start()
0809 07:56:06.264775 (+ 3629us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:56:06.265773 (+ 998us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
0809 07:56:06.265781 (+ 8us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
0809 07:56:06.265785 (+ 4us) server_negotiation.cc:968] calling sasl_server_step()
0809 07:56:06.265875 (+ 90us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0809 07:56:06.266345 (+ 470us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
0809 07:56:06.266349 (+ 4us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
0809 07:56:06.266351 (+ 2us) server_negotiation.cc:968] calling sasl_server_step()
0809 07:56:06.266619 (+ 268us) server_negotiation.cc:378] Sending SASL_SUCCESS NegotiatePB response
0809 07:56:06.266770 (+ 151us) server_negotiation.cc:1036] Waiting for connection context
0809 07:56:06.296402 (+ 29632us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:47003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 07:56:06.299573 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13698: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:38983: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:56:06.299705 14095 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:06.298137 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:40698 (local address 127.26.37.62:38983)
0809 07:56:06.298296 (+ 159us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:06.298300 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:06.298369 (+ 69us) server_negotiation.cc:408] Connection header received
0809 07:56:06.298539 (+ 170us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:06.298544 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:06.298657 (+ 113us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:06.298757 (+ 100us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:06.299496 (+ 739us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40698: BlockingRecv error: recv got EOF from 127.0.0.1:40698 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:56:06.299746 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13698
I20260809 07:56:06.328182 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12688
I20260809 07:56:06.329313 12774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:06.441242 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12688
I20260809 07:56:06.472631 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 12796
I20260809 07:56:06.473965 12892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:06.605746 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 12796
I20260809 07:56:06.643543 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 13217
I20260809 07:56:06.644909 13460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:06.763352 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 13217
[ OK ] ExternalMiniClusterTest.TestAddMaster/6 (10041 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/7
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:56:06 dist-test-slave-bw88 krb5kdc[14269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:56:06 dist-test-slave-bw88 krb5kdc[14269](info): set up 2 sockets
Aug 09 07:56:06 dist-test-slave-bw88 krb5kdc[14269](info): commencing operation
krb5kdc: starting...
W20260809 07:56:08.994731 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.189s user 0.007s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:56:09 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:56:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:56:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:09.386634 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41657
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:09.759464 14296 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:09.759845 14296 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:09.760010 14296 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:09.768338 14296 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:09.768498 14296 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:09.768622 14296 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:09.768707 14296 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:09.779196 14296 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:09.780854 14296 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:09.782326 14296 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:09.790524 14301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:09.790552 14303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:09.791548 14306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:09.791946 14296 server_base.cc:1061] running on GCE node
I20260809 07:56:09.792670 14296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:09.794126 14296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:09.795328 14296 hybrid_clock.cc:648] HybridClock initialized: now 1786262169795290 us; error 33 us; skew 500 ppm
I20260809 07:56:09.795699 14296 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:09 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:09.799120 14296 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:56:09.800637 14296 webserver.cc:533] Webserver started at http://127.26.37.62:33735/ using document root <none> and password file <none>
I20260809 07:56:09.801388 14296 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:09.801517 14296 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:09.801827 14296 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:09.804132 14296 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "62e39be8bee841539b8d43ab0436dae3"
format_stamp: "Formatted at 2026-08-09 07:56:09 on dist-test-slave-bw88"
I20260809 07:56:09.804850 14296 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "62e39be8bee841539b8d43ab0436dae3"
format_stamp: "Formatted at 2026-08-09 07:56:09 on dist-test-slave-bw88"
I20260809 07:56:09.810089 14296 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260809 07:56:09.813622 14318 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:09.814945 14296 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:09.815109 14296 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "62e39be8bee841539b8d43ab0436dae3"
format_stamp: "Formatted at 2026-08-09 07:56:09 on dist-test-slave-bw88"
I20260809 07:56:09.815265 14296 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:09.848408 14296 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:09.852259 14296 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:09.852516 14296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:09.873935 14296 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41657
I20260809 07:56:09.873986 14388 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41657 every 8 connection(s)
I20260809 07:56:09.875656 14296 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:56:09.879806 14389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:09.880515 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14296
I20260809 07:56:09.880736 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:56:09.888671 14389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:09 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:09.912660 14389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:09.914423 14324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35281: connect: Connection refused (error 111)
W20260809 07:56:09.916260 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35281: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35281: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:09.965629 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } attempt: 1
W20260809 07:56:09.967941 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35281: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35281: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:10.030939 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:35281
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:10.036185 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } attempt: 2
W20260809 07:56:10.038319 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35281: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35281: connect: Connection refused (error 111)
I20260809 07:56:10.129593 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } attempt: 3
W20260809 07:56:10.131803 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35281: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35281: connect: Connection refused (error 111)
I20260809 07:56:10.275168 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } attempt: 4
W20260809 07:56:10.277639 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:35281: Network error: Client connection negotiation failed: client connection to 127.26.37.61:35281: connect: Connection refused (error 111)
W20260809 07:56:10.381273 14400 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:10.381665 14400 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:10.381760 14400 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:10.389866 14400 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:10.389962 14400 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:10.390018 14400 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:10.390066 14400 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:10.400435 14400 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:35281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:10.402016 14400 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:10.403322 14400 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:10.411855 14411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:10.412248 14412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:10.412622 14414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:10.413522 14400 server_base.cc:1061] running on GCE node
I20260809 07:56:10.414249 14400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:10.415571 14400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:10.416774 14400 hybrid_clock.cc:648] HybridClock initialized: now 1786262170416733 us; error 27 us; skew 500 ppm
I20260809 07:56:10.417109 14400 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:10 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262170, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:10.420317 14400 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:56:10.421948 14400 webserver.cc:533] Webserver started at http://127.26.37.61:41893/ using document root <none> and password file <none>
I20260809 07:56:10.422613 14400 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:10.422713 14400 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:10.422998 14400 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:10.425262 14400 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "ace6c8ef84a0498baf696b53b2ce87d4"
format_stamp: "Formatted at 2026-08-09 07:56:10 on dist-test-slave-bw88"
I20260809 07:56:10.426136 14400 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "ace6c8ef84a0498baf696b53b2ce87d4"
format_stamp: "Formatted at 2026-08-09 07:56:10 on dist-test-slave-bw88"
I20260809 07:56:10.431667 14400 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 07:56:10.435091 14425 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:10.436378 14400 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:56:10.436525 14400 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "ace6c8ef84a0498baf696b53b2ce87d4"
format_stamp: "Formatted at 2026-08-09 07:56:10 on dist-test-slave-bw88"
I20260809 07:56:10.436688 14400 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:10.459393 14400 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:10.468201 14400 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:10.468421 14400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:10.488868 14400 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:35281
I20260809 07:56:10.488888 14492 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:35281 every 8 connection(s)
I20260809 07:56:10.490422 14400 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:56:10.494710 14493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:10.495375 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14400
I20260809 07:56:10.495575 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:56:10.500540 14493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:10 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262170, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:10.526513 14493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:10 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262170, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
I20260809 07:56:10.544371 14493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
W20260809 07:56:10.545603 14427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
W20260809 07:56:10.547250 14493 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
I20260809 07:56:10.576908 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } attempt: 5
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Aug 09 07:56:10 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:10.590864 14389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:10.592839 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
I20260809 07:56:10.596657 14493 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 1
W20260809 07:56:10.598518 14493 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:56:10.644107 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 1
W20260809 07:56:10.646667 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:10.660881 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:34873
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:10.666756 14493 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 2
W20260809 07:56:10.668874 14493 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
I20260809 07:56:10.714946 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 2
W20260809 07:56:10.717087 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
I20260809 07:56:10.760179 14493 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 3
W20260809 07:56:10.762473 14493 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
I20260809 07:56:10.823324 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 3
W20260809 07:56:10.825418 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
I20260809 07:56:10.905732 14493 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 4
W20260809 07:56:10.907856 14493 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
I20260809 07:56:11.002640 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 4
W20260809 07:56:11.004812 14389 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34873: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34873: connect: Connection refused (error 111)
W20260809 07:56:11.017908 14512 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:11.018237 14512 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:11.018391 14512 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:11.026731 14512 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:11.026811 14512 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:11.026865 14512 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:11.026938 14512 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:11.037289 14512 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:34873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:11.038894 14512 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:11.040246 14512 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:11.048301 14522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:11.048408 14521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:11.049165 14524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:11.049836 14512 server_base.cc:1061] running on GCE node
I20260809 07:56:11.050644 14512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:11.052054 14512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:11.053287 14512 hybrid_clock.cc:648] HybridClock initialized: now 1786262171053259 us; error 52 us; skew 500 ppm
I20260809 07:56:11.053649 14512 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262171, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:11.056842 14512 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:56:11.058355 14512 webserver.cc:533] Webserver started at http://127.26.37.60:46791/ using document root <none> and password file <none>
I20260809 07:56:11.059037 14512 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:11.059128 14512 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:11.059407 14512 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:11.061781 14512 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "6a91d93684de436aa16f29de26a05a56"
format_stamp: "Formatted at 2026-08-09 07:56:11 on dist-test-slave-bw88"
I20260809 07:56:11.062505 14512 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "6a91d93684de436aa16f29de26a05a56"
format_stamp: "Formatted at 2026-08-09 07:56:11 on dist-test-slave-bw88"
I20260809 07:56:11.067653 14512 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 07:56:11.070767 14534 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:11.072007 14512 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:56:11.072186 14512 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "6a91d93684de436aa16f29de26a05a56"
format_stamp: "Formatted at 2026-08-09 07:56:11 on dist-test-slave-bw88"
I20260809 07:56:11.072343 14512 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:11.090987 14512 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:11.096175 14512 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:11.096488 14512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:11.116397 14512 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:34873
I20260809 07:56:11.116417 14602 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:34873 every 8 connection(s)
I20260809 07:56:11.117978 14512 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:56:11.122350 14604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:11.126538 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14512
I20260809 07:56:11.126760 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:56:11.129124 14604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262171, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:11.163555 14604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262171, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:11.179975 14604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262171, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:11.197450 14604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: Bootstrap starting.
I20260809 07:56:11.200549 14604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:11.201843 14604 log.cc:826] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:11.204334 14604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: No bootstrap required, opened a new log
I20260809 07:56:11.207094 14493 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 5
I20260809 07:56:11.210889 14604 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.211180 14604 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:11.211309 14604 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a91d93684de436aa16f29de26a05a56, State: Initialized, Role: FOLLOWER
I20260809 07:56:11.212042 14604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.213279 14619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.213620 14619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:11.214424 14604 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262170, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:11.227239 14493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4: Bootstrap starting.
W20260809 07:56:11.228223 14633 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:11.228396 14633 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:11.231336 14493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:11.225194 14512 master_runner.cc:187] Error getting master registration for 127.26.37.62:41657: OK, instance_id { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" instance_seqno: 1786262169868007 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:56:11.231982 14512 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:56:11.232751 14493 log.cc:826] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:11.235229 14493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4: No bootstrap required, opened a new log
I20260809 07:56:11.240025 14493 raft_consensus.cc:359] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.240307 14493 raft_consensus.cc:385] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:11.240485 14493 raft_consensus.cc:740] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace6c8ef84a0498baf696b53b2ce87d4, State: Initialized, Role: FOLLOWER
I20260809 07:56:11.241088 14493 consensus_queue.cc:260] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.242121 14636 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.242446 14636 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:11.243213 14493 sys_catalog.cc:565] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:56:11.254838 14649 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:11.255070 14649 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:11.251981 14400 master_runner.cc:187] Error getting master registration for 127.26.37.62:41657: OK, instance_id { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" instance_seqno: 1786262169868007 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:56:11.258867 14400 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:56:11.266986 14636 raft_consensus.cc:493] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:56:11.267244 14636 raft_consensus.cc:515] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.269073 14636 leader_election.cc:290] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62e39be8bee841539b8d43ab0436dae3 (127.26.37.62:41657), 6a91d93684de436aa16f29de26a05a56 (127.26.37.60:34873)
I20260809 07:56:11.270026 14569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a91d93684de436aa16f29de26a05a56" is_pre_election: true
I20260809 07:56:11.270593 14569 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace6c8ef84a0498baf696b53b2ce87d4 in term 0.
I20260809 07:56:11.271803 14427 leader_election.cc:304] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a91d93684de436aa16f29de26a05a56, ace6c8ef84a0498baf696b53b2ce87d4; no voters:
I20260809 07:56:11.272210 14636 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:56:11.272372 14636 raft_consensus.cc:493] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:56:11.272467 14636 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:11.276062 14636 raft_consensus.cc:515] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.277631 14636 leader_election.cc:290] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [CANDIDATE]: Term 1 election: Requested vote from peers 62e39be8bee841539b8d43ab0436dae3 (127.26.37.62:41657), 6a91d93684de436aa16f29de26a05a56 (127.26.37.60:34873)
I20260809 07:56:11.278419 14569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a91d93684de436aa16f29de26a05a56"
I20260809 07:56:11.278612 14569 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:11.282022 14389 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } attempt: 5
Aug 09 07:56:11 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:11.293215 14569 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace6c8ef84a0498baf696b53b2ce87d4 in term 1.
I20260809 07:56:11.294445 14427 leader_election.cc:304] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a91d93684de436aa16f29de26a05a56, ace6c8ef84a0498baf696b53b2ce87d4; no voters:
I20260809 07:56:11.297024 14636 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:56:11.310387 14636 raft_consensus.cc:697] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 1 LEADER]: Becoming Leader. State: Replica: ace6c8ef84a0498baf696b53b2ce87d4, State: Running, Role: LEADER
I20260809 07:56:11.311110 14636 consensus_queue.cc:237] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.311671 14389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: Bootstrap starting.
I20260809 07:56:11.316141 14389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:11.317618 14389 log.cc:826] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:11.320873 14651 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ace6c8ef84a0498baf696b53b2ce87d4. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.321659 14651 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:11.327199 14389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: No bootstrap required, opened a new log
I20260809 07:56:11.334043 14655 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:56:11.336455 14655 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:56:11.342041 14389 raft_consensus.cc:359] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.342382 14389 raft_consensus.cc:385] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:11.342510 14389 raft_consensus.cc:740] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e39be8bee841539b8d43ab0436dae3, State: Initialized, Role: FOLLOWER
I20260809 07:56:11.343343 14389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } }
I20260809 07:56:11.347368 14659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
W20260809 07:56:11.348788 14541 tablet.cc:2406] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:56:11.350298 14569 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 1 FOLLOWER]: Refusing update from remote peer ace6c8ef84a0498baf696b53b2ce87d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:56:11.353194 14651 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:56:11.347645 14659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:11.354969 14389 sys_catalog.cc:565] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:56:11.355515 14329 tablet.cc:2406] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:56:11.358738 14354 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:11.361572 14355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e39be8bee841539b8d43ab0436dae3"
I20260809 07:56:11.362684 14356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e39be8bee841539b8d43ab0436dae3" is_pre_election: true
I20260809 07:56:11.365401 14354 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 1 FOLLOWER]: Refusing update from remote peer ace6c8ef84a0498baf696b53b2ce87d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:56:11.366214 14651 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:56:11.405562 14619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ace6c8ef84a0498baf696b53b2ce87d4. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.405826 14619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:11.409538 14658 mvcc.cc:204] Tried to move back new op lower bound from 7316529853831348224 to 7316529853696536576. Current Snapshot: MvccSnapshot[applied={T|T < 7316529853831348224}]
I20260809 07:56:11.413190 14662 mvcc.cc:204] Tried to move back new op lower bound from 7316529853831348224 to 7316529853696536576. Current Snapshot: MvccSnapshot[applied={T|T < 7316529853831348224}]
I20260809 07:56:11.415932 14659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ace6c8ef84a0498baf696b53b2ce87d4. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.416213 14659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:11.418812 14619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.419010 14619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:11.431936 14665 mvcc.cc:204] Tried to move back new op lower bound from 7316529853831348224 to 7316529853696536576. Current Snapshot: MvccSnapshot[applied={T|T < 7316529853831348224}]
I20260809 07:56:11.442716 14636 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.442999 14636 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:11.443662 14636 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.443878 14636 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:11.445055 14659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } }
I20260809 07:56:11.445266 14659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:11.446851 14655 catalog_manager.cc:1383] Generated new cluster ID: 4d79720165434a989cb9bfa69dcb2c19
I20260809 07:56:11.446952 14655 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:56:11.481389 14655 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:56:11.483230 14655 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:56:11.492549 14685 catalog_manager.cc:1295] Loaded cluster ID: 4d79720165434a989cb9bfa69dcb2c19
I20260809 07:56:11.492654 14685 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: loading cluster ID for follower catalog manager: success
I20260809 07:56:11.496591 14685 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: acquiring CA information for follower catalog manager: success
W20260809 07:56:11.497917 14685 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260809 07:56:11.520176 14655 catalog_manager.cc:6092] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4: Generated new TSK 0
I20260809 07:56:11.521471 14655 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
W20260809 07:56:11.774488 26772 mini_kdc.cc:268] Time spent creating service keytab for HTTP/127.26.37.1: real 0.136s user 0.000s sys 0.032s
I20260809 07:56:11.775556 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:12.178376 14695 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:12.178646 14695 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:12.178730 14695 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:12.187134 14695 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:12.187368 14695 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:12.197417 14695 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:12.198956 14695 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:12.200296 14695 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:12.210350 14701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:12.210381 14700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:12.210609 14706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:12.211614 14695 server_base.cc:1061] running on GCE node
I20260809 07:56:12.212100 14695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:12.213083 14695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:12.214290 14695 hybrid_clock.cc:648] HybridClock initialized: now 1786262172214251 us; error 28 us; skew 500 ppm
I20260809 07:56:12.214557 14695 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:12 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:12.217847 14695 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:12.219293 14695 webserver.cc:533] Webserver started at http://127.26.37.1:33531/ using document root <none> and password file <none>
I20260809 07:56:12.219928 14695 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:12.220094 14695 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:12.220393 14695 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:12.222908 14695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "069c89cf557144c7a26850fd845c4668"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:12.223758 14695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "069c89cf557144c7a26850fd845c4668"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:12.228873 14695 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:56:12.230394 14633 catalog_manager.cc:1295] Loaded cluster ID: 4d79720165434a989cb9bfa69dcb2c19
I20260809 07:56:12.230495 14633 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: loading cluster ID for follower catalog manager: success
I20260809 07:56:12.232503 14716 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:12.233634 14633 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: acquiring CA information for follower catalog manager: success
I20260809 07:56:12.233891 14695 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:56:12.234057 14695 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "069c89cf557144c7a26850fd845c4668"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:12.234241 14695 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:12.235419 14633 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:56:12.252629 14695 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:12.256382 14695 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:12.256706 14695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:12.257740 14695 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:12.259253 14695 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:12.259374 14695 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:12.259517 14695 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:12.259608 14695 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:12.295113 14695 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43689
I20260809 07:56:12.295199 14871 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43689 every 8 connection(s)
I20260809 07:56:12.296504 14695 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:12.301427 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14695
I20260809 07:56:12.301676 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:56:12 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:12 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:12 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:12.341707 14876 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35281
I20260809 07:56:12.341768 14875 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41657
I20260809 07:56:12.342203 14875 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:12.342212 14876 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:12.342264 14872 heartbeater.cc:344] Connected to a master server at 127.26.37.60:34873
I20260809 07:56:12.342474 14872 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:12.343466 14876 heartbeater.cc:507] Master 127.26.37.61:35281 requested a full tablet report, sending...
I20260809 07:56:12.343506 14875 heartbeater.cc:507] Master 127.26.37.62:41657 requested a full tablet report, sending...
I20260809 07:56:12.343479 14872 heartbeater.cc:507] Master 127.26.37.60:34873 requested a full tablet report, sending...
I20260809 07:56:12.346238 14448 ts_manager.cc:194] Registered new tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:12.346315 14345 ts_manager.cc:194] Registered new tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:12.346338 14555 ts_manager.cc:194] Registered new tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:12.348330 14448 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:41509
I20260809 07:56:12.352854 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:12.498971 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:40947
--webserver_interface=127.26.37.59
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:12.500825 14685 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:56:12.853305 14892 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:12.853778 14892 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:12.853935 14892 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:12.862375 14892 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:12.862502 14892 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:12.862622 14892 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:12.862702 14892 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:12.873225 14892 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:40947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:12.874902 14892 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:12.876292 14892 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:12.884831 14900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:12.884869 14899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:12.885671 14904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:12.886143 14892 server_base.cc:1061] running on GCE node
I20260809 07:56:12.886829 14892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:12.888163 14892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:12.889420 14892 hybrid_clock.cc:648] HybridClock initialized: now 1786262172889366 us; error 34 us; skew 500 ppm
I20260809 07:56:12.889726 14892 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:12 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:12.892903 14892 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:56:12.894532 14892 webserver.cc:533] Webserver started at http://127.26.37.59:39063/ using document root <none> and password file <none>
I20260809 07:56:12.895198 14892 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:12.895308 14892 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:12.895591 14892 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:12.898054 14892 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "61b1ab2ca3d04f4c91bccf85b8513abf"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:12.898811 14892 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "61b1ab2ca3d04f4c91bccf85b8513abf"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:12.904150 14892 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:56:12.907505 14912 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:12.908782 14892 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:56:12.908993 14892 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "61b1ab2ca3d04f4c91bccf85b8513abf"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:12.909225 14892 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:12.925473 14892 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:12.929328 14892 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:12.929668 14892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:12.951459 14892 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:40947
I20260809 07:56:12.951535 14985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:40947 every 8 connection(s)
I20260809 07:56:12.953137 14892 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:56:12.957628 14986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:12.962472 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 14892
I20260809 07:56:12.964512 14986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:12 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:12 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:12.985004 14986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:12 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:12.998988 14986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:13 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:13.011933 14986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:13 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262172, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:13.029307 14986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Bootstrap starting.
I20260809 07:56:13.032577 14986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:13.033703 14986 log.cc:826] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:13.036240 14986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: No bootstrap required, opened a new log
I20260809 07:56:13.041935 14986 raft_consensus.cc:359] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } }
I20260809 07:56:13.042233 14986 raft_consensus.cc:385] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:13.042310 14986 raft_consensus.cc:740] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b1ab2ca3d04f4c91bccf85b8513abf, State: Initialized, Role: FOLLOWER
I20260809 07:56:13.042845 14986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } }
I20260809 07:56:13.043679 14995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } } }
I20260809 07:56:13.044055 14995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:13.044793 14986 sys_catalog.cc:565] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:56:13.054225 15009 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:13.054409 15009 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:13.061420 14892 master_runner.cc:429] Detected that this master 61b1ab2ca3d04f4c91bccf85b8513abf is joining an existing cluster
I20260809 07:56:13.061549 14892 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:56:13.062538 14892 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:56:13.062896 14892 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:56:13.063103 14892 tablet_replica.cc:333] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: stopping tablet replica
I20260809 07:56:13.064651 14892 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:40947
Aug 09 07:56:13 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262170, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:13.120410 14448 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:40947: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 40947
}
attrs {
promote: true
}
}
dest_uuid: "ace6c8ef84a0498baf696b53b2ce87d4"
cas_config_opid_index: -1
I20260809 07:56:13.121866 14448 consensus_queue.cc:237] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } }
I20260809 07:56:13.125407 14354 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 1 FOLLOWER]: Refusing update from remote peer ace6c8ef84a0498baf696b53b2ce87d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260809 07:56:13.126324 14426 consensus_peers.cc:597] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 -> Peer 61b1ab2ca3d04f4c91bccf85b8513abf (127.26.37.59:40947): Couldn't send request to peer 61b1ab2ca3d04f4c91bccf85b8513abf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:56:13.126542 14917 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:56:13.128098 14569 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 1 FOLLOWER]: Refusing update from remote peer ace6c8ef84a0498baf696b53b2ce87d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:56:13.128742 14917 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:56:13.135983 15031 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 61b1ab2ca3d04f4c91bccf85b8513abf (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.136706 14354 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 61b1ab2ca3d04f4c91bccf85b8513abf (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.140890 15031 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:40947
I20260809 07:56:13.141793 14892 master.cc:562] Master@127.26.37.59:40947 shutting down...
I20260809 07:56:13.144254 15031 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.144523 15031 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:13.144434 15034 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.144939 15034 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:13.147059 14917 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.147351 14917 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:13.148197 14569 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 61b1ab2ca3d04f4c91bccf85b8513abf (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.156711 15035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.157553 15035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:13.157644 15040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.157954 15040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:13.198741 14892 master.cc:584] Master@127.26.37.59:40947 shutdown complete.
I20260809 07:56:13.198827 14892 master_runner.cc:306] Clearing existing system tablet
I20260809 07:56:13.199218 14892 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:13.199326 14892 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:13.202077 14892 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 07:56:13.203686 15043 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:13.204057 14892 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:13.204316 14892 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 07:56:13.204408 14892 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "61b1ab2ca3d04f4c91bccf85b8513abf"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:13.209216 14892 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:56:13.216668 14892 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:56:13.216846 14892 log.cc:1199] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:56:13.217414 14892 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Deleting consensus metadata
I20260809 07:56:13.217732 14892 master_runner.cc:316] Copying system tablet from 127.26.37.61:35281
I20260809 07:56:13.220762 14892 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.61:35281
I20260809 07:56:13.234977 14474 tablet_copy_service.cc:140] P ace6c8ef84a0498baf696b53b2ce87d4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 61b1ab2ca3d04f4c91bccf85b8513abf ({username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:60724)
I20260809 07:56:13.235359 14474 tablet_copy_service.cc:161] P ace6c8ef84a0498baf696b53b2ce87d4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 61b1ab2ca3d04f4c91bccf85b8513abf at {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:60724: session id = 61b1ab2ca3d04f4c91bccf85b8513abf-00000000000000000000000000000000
I20260809 07:56:13.239869 14474 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:56:13.242456 14892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:56:13.249724 14892 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: tablet copy: Starting download of 0 data blocks...
I20260809 07:56:13.250056 14892 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: tablet copy: Starting download of 1 WAL segments...
I20260809 07:56:13.252779 14892 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:56:13.257624 14474 tablet_copy_service.cc:342] P ace6c8ef84a0498baf696b53b2ce87d4: Request end of tablet copy session 61b1ab2ca3d04f4c91bccf85b8513abf-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:60724
I20260809 07:56:13.257848 14474 tablet_copy_service.cc:434] P ace6c8ef84a0498baf696b53b2ce87d4: ending tablet copy session 61b1ab2ca3d04f4c91bccf85b8513abf-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 61b1ab2ca3d04f4c91bccf85b8513abf
W20260809 07:56:13.266157 14892 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260809 07:56:13.270512 14892 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:13.273459 15055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:13.274001 15053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:13.278194 15058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:13.278221 14892 server_base.cc:1061] running on GCE node
I20260809 07:56:13.278702 14892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:13.279438 14892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:13.280669 14892 hybrid_clock.cc:648] HybridClock initialized: now 1786262173280612 us; error 39 us; skew 500 ppm
I20260809 07:56:13.280929 14892 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:13 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262173, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:13.283120 14892 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:56:13.284565 14892 webserver.cc:533] Webserver started at http://127.26.37.59:44587/ using document root <none> and password file <none>
I20260809 07:56:13.284871 14892 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:13.284976 14892 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:13.287213 14892 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 07:56:13.289112 15065 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:13.289644 14892 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 07:56:13.289778 14892 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "61b1ab2ca3d04f4c91bccf85b8513abf"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:13.289960 14892 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:13.305518 14892 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:13.309278 14892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:13.329069 14892 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:40947
I20260809 07:56:13.329092 15135 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:40947 every 8 connection(s)
I20260809 07:56:13.330456 14892 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:56:13.334022 15136 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:56:13.336176 15136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Bootstrap starting.
I20260809 07:56:13.351354 14876 heartbeater.cc:499] Master 127.26.37.61:35281 was elected leader, sending a full tablet report...
I20260809 07:56:13.352502 15136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:56:13.353070 15136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: Bootstrap complete.
I20260809 07:56:13.354266 15136 raft_consensus.cc:359] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } }
I20260809 07:56:13.354527 15136 raft_consensus.cc:740] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 61b1ab2ca3d04f4c91bccf85b8513abf, State: Initialized, Role: LEARNER
I20260809 07:56:13.354801 15136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } }
I20260809 07:56:13.355885 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.356246 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: This master's current role is: LEARNER
I20260809 07:56:13.356244 15136 sys_catalog.cc:565] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:13.365257 15162 catalog_manager.cc:1295] Loaded cluster ID: 4d79720165434a989cb9bfa69dcb2c19
I20260809 07:56:13.365386 15162 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: loading cluster ID for follower catalog manager: success
I20260809 07:56:13.367959 15162 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: acquiring CA information for follower catalog manager: success
I20260809 07:56:13.369860 15162 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:56:13.411518 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14695
Aug 09 07:56:13 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:56:13.431627 14867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:13.523978 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14695
I20260809 07:56:13.565819 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43689
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--webserver_port=33531
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:13.613045 15101 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:56:13.613749 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ace6c8ef84a0498baf696b53b2ce87d4. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:13.614002 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: This master's current role is: LEARNER
W20260809 07:56:13.926474 15166 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:13.926783 15166 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:13.926884 15166 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:13.936998 15166 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:13.937186 15166 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:13.947731 15166 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33531
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:13.949431 15166 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:13.950788 15166 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:13.960407 15173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:13.960430 15174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:13.961670 15176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:13.961717 15166 server_base.cc:1061] running on GCE node
I20260809 07:56:13.962368 15166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:13.963291 15166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:13.964514 15166 hybrid_clock.cc:648] HybridClock initialized: now 1786262173964468 us; error 33 us; skew 500 ppm
I20260809 07:56:13.964816 15166 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:13 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262173, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:13.968101 15166 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:13.969625 15166 webserver.cc:533] Webserver started at http://127.26.37.1:33531/ using document root <none> and password file <none>
I20260809 07:56:13.970264 15166 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:13.970376 15166 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:13.975629 15166 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 07:56:13.978430 15184 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:13.979768 15166 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:13.979935 15166 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "069c89cf557144c7a26850fd845c4668"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:13.980726 15166 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:14.012288 15166 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:14.016189 15166 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:14.016476 15166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:14.017657 15166 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:14.019342 15166 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:14.019457 15166 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:14.019580 15166 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:14.019677 15166 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:14.053534 15166 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43689
I20260809 07:56:14.053575 15341 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43689 every 8 connection(s)
I20260809 07:56:14.055380 15166 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:14.062700 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15166
Aug 09 07:56:14 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262173, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:14.092103 15344 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41657
I20260809 07:56:14.092566 15344 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:14.093613 15344 heartbeater.cc:507] Master 127.26.37.62:41657 requested a full tablet report, sending...
Aug 09 07:56:14 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262173, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:14.096429 14345 ts_manager.cc:194] Re-registered known tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
Aug 09 07:56:14 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262173, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:56:14 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262173, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:14.102793 15033 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: true } } }
I20260809 07:56:14.103266 15033 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:14.104738 15358 raft_consensus.cc:1064] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4: attempting to promote NON_VOTER 61b1ab2ca3d04f4c91bccf85b8513abf to VOTER
I20260809 07:56:14.105844 15358 consensus_queue.cc:237] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } }
I20260809 07:56:14.107679 15343 heartbeater.cc:344] Connected to a master server at 127.26.37.60:34873
I20260809 07:56:14.107857 15343 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:14.108398 15343 heartbeater.cc:507] Master 127.26.37.60:34873 requested a full tablet report, sending...
I20260809 07:56:14.108933 14569 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 1 FOLLOWER]: Refusing update from remote peer ace6c8ef84a0498baf696b53b2ce87d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:14.110157 14554 ts_manager.cc:194] Re-registered known tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:14.110657 15033 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:14.111706 14354 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 1 FOLLOWER]: Refusing update from remote peer ace6c8ef84a0498baf696b53b2ce87d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:14.113140 15345 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35281
I20260809 07:56:14.113293 15345 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:14.113416 15342 heartbeater.cc:344] Connected to a master server at 127.26.37.59:40947
I20260809 07:56:14.113551 15342 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:14.113924 15345 heartbeater.cc:507] Master 127.26.37.61:35281 requested a full tablet report, sending...
I20260809 07:56:14.114010 15342 heartbeater.cc:507] Master 127.26.37.59:40947 requested a full tablet report, sending...
I20260809 07:56:14.115118 15033 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:14.115655 14448 ts_manager.cc:194] Re-registered known tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:14.116120 15090 ts_manager.cc:194] Registered new tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:14.116532 14448 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:46221
I20260809 07:56:14.119017 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:14.119764 15358 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 61b1ab2ca3d04f4c91bccf85b8513abf (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.121238 14354 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 61b1ab2ca3d04f4c91bccf85b8513abf (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.124101 15361 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.124293 15361 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:14.124588 15361 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.124768 15361 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:14.124536 14569 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 61b1ab2ca3d04f4c91bccf85b8513abf (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.125051 15358 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.125267 15358 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:14.126315 15101 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 1 LEARNER]: Refusing update from remote peer ace6c8ef84a0498baf696b53b2ce87d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:14.128945 15361 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:14.130023 15364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.130334 15364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a91d93684de436aa16f29de26a05a56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:14.132673 15101 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 61b1ab2ca3d04f4c91bccf85b8513abf (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.134581 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:14.134747 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14296
I20260809 07:56:14.135677 14382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:14.138439 15368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.138685 15368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b1ab2ca3d04f4c91bccf85b8513abf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:14.246374 15363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.246719 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14296
I20260809 07:56:14.246709 15363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:14.278733 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41657
--webserver_interface=127.26.37.62
--webserver_port=33735
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:14.573767 14432 consensus_peers.cc:597] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 -> Peer 62e39be8bee841539b8d43ab0436dae3 (127.26.37.62:41657): Couldn't send request to peer 62e39be8bee841539b8d43ab0436dae3. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:41657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:56:14.590669 15358 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.591218 15358 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
W20260809 07:56:14.649401 15369 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:14.649765 15369 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:14.649920 15369 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:14.658571 15369 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:14.658665 15369 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:14.658779 15369 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:14.658871 15369 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:14.669766 15369 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=33735
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:14.671398 15369 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:14.672797 15369 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:14.681282 15377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:14.681586 15376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:14.681945 15379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:14.682400 15369 server_base.cc:1061] running on GCE node
I20260809 07:56:14.683182 15369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:14.684535 15369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:14.685783 15369 hybrid_clock.cc:648] HybridClock initialized: now 1786262174685727 us; error 42 us; skew 500 ppm
I20260809 07:56:14.686200 15369 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:14 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262174, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:14.689556 15369 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:56:14.691207 15369 webserver.cc:533] Webserver started at http://127.26.37.62:33735/ using document root <none> and password file <none>
I20260809 07:56:14.691959 15369 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:14.692128 15369 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:14.697568 15369 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:14.700403 15388 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:14.701740 15369 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:14.701934 15369 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "62e39be8bee841539b8d43ab0436dae3"
format_stamp: "Formatted at 2026-08-09 07:56:09 on dist-test-slave-bw88"
I20260809 07:56:14.702641 15369 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:14.717015 15369 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:14.720981 15369 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:14.721274 15369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:14.742616 15369 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41657
I20260809 07:56:14.742627 15469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41657 every 8 connection(s)
I20260809 07:56:14.744234 15369 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:56:14.751497 15470 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:56:14.752807 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15369
I20260809 07:56:14.753129 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15166
I20260809 07:56:14.755507 15470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: Bootstrap starting.
I20260809 07:56:14.771368 15337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:14.777248 15470 log.cc:826] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:14.790287 15470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:56:14.790922 15470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: Bootstrap complete.
I20260809 07:56:14.801388 15470 raft_consensus.cc:359] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } }
I20260809 07:56:14.802318 15470 raft_consensus.cc:740] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e39be8bee841539b8d43ab0436dae3, State: Initialized, Role: FOLLOWER
I20260809 07:56:14.803166 15470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } }
I20260809 07:56:14.804019 15476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:14.804314 15476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:14.805270 15470 sys_catalog.cc:565] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:14.816951 15489 catalog_manager.cc:1295] Loaded cluster ID: 4d79720165434a989cb9bfa69dcb2c19
I20260809 07:56:14.817068 15489 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: loading cluster ID for follower catalog manager: success
I20260809 07:56:14.824504 15489 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: acquiring CA information for follower catalog manager: success
I20260809 07:56:14.827544 15489 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:56:14 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262174, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:14.905575 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15166
I20260809 07:56:14.932999 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43689
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--webserver_port=33531
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:15.161218 15358 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:15.161567 15358 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:15.174065 15476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ace6c8ef84a0498baf696b53b2ce87d4. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:15.174413 15476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e39be8bee841539b8d43ab0436dae3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:56:15.339548 15497 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:15.339959 15497 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:15.340134 15497 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:15.348758 15497 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:15.349006 15497 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:15.360000 15497 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33531
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:15.361779 15497 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:15.363129 15497 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:15.372912 15508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:15.373099 15510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:15.373055 15506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:15.373976 15497 server_base.cc:1061] running on GCE node
I20260809 07:56:15.374529 15497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:15.375401 15497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:15.376654 15497 hybrid_clock.cc:648] HybridClock initialized: now 1786262175376620 us; error 54 us; skew 500 ppm
I20260809 07:56:15.377030 15497 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:15 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262175, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:15.380448 15497 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:15.382079 15497 webserver.cc:533] Webserver started at http://127.26.37.1:33531/ using document root <none> and password file <none>
I20260809 07:56:15.382809 15497 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:15.382958 15497 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:15.388341 15497 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:56:15.391027 15520 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:15.392370 15497 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:56:15.392619 15497 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "069c89cf557144c7a26850fd845c4668"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:15.393314 15497 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:15.423345 15497 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:15.427331 15497 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:15.427613 15497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:15.428623 15497 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:15.430267 15497 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:15.430394 15497 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:15.430538 15497 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:15.430641 15497 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:15.466343 15497 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43689
I20260809 07:56:15.466395 15678 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43689 every 8 connection(s)
I20260809 07:56:15.467953 15497 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:15.472476 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15497
Aug 09 07:56:15 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262175, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:56:15 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262175, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:15 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262175, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:15.524470 15680 heartbeater.cc:344] Connected to a master server at 127.26.37.59:40947
I20260809 07:56:15.524570 15687 heartbeater.cc:344] Connected to a master server at 127.26.37.61:35281
I20260809 07:56:15.524909 15687 heartbeater.cc:461] Registering TS with master...
Aug 09 07:56:15 dist-test-slave-bw88 krb5kdc[14269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262175, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:15.524916 15680 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:15.526135 15680 heartbeater.cc:507] Master 127.26.37.59:40947 requested a full tablet report, sending...
I20260809 07:56:15.527702 15687 heartbeater.cc:507] Master 127.26.37.61:35281 requested a full tablet report, sending...
I20260809 07:56:15.529829 15090 ts_manager.cc:194] Re-registered known tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:15.531886 14448 ts_manager.cc:194] Re-registered known tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:15.533102 14448 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:60607
I20260809 07:56:15.538523 15681 heartbeater.cc:344] Connected to a master server at 127.26.37.60:34873
I20260809 07:56:15.538690 15681 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:15.539155 15681 heartbeater.cc:507] Master 127.26.37.60:34873 requested a full tablet report, sending...
I20260809 07:56:15.540145 15686 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41657
I20260809 07:56:15.540288 15686 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:15.540360 14554 ts_manager.cc:194] Re-registered known tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:15.540560 15686 heartbeater.cc:507] Master 127.26.37.62:41657 requested a full tablet report, sending...
I20260809 07:56:15.542579 15414 ts_manager.cc:194] Registered new tserver with Master: 069c89cf557144c7a26850fd845c4668 (127.26.37.1:43689)
I20260809 07:56:15.575028 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15497
W20260809 07:56:15.576427 15702 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:15.572601 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49652 (local address 127.26.37.1:43689)
0809 07:56:15.573452 (+ 851us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:15.573459 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:15.573494 (+ 35us) server_negotiation.cc:408] Connection header received
0809 07:56:15.573562 (+ 68us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:15.573569 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:15.573788 (+ 219us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:15.573975 (+ 187us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:15.574861 (+ 886us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49652: BlockingRecv error: recv got EOF from 127.0.0.1:49652 (error 108)
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":481,"threads_started":1}
W20260809 07:56:15.577534 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15497: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:43689: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:15.577665 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15497
I20260809 07:56:15.605290 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43689
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--webserver_port=33531
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:15.688757 15498 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "62e39be8bee841539b8d43ab0436dae3" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41657 } } peers { permanent_uuid: "ace6c8ef84a0498baf696b53b2ce87d4" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 35281 } } peers { permanent_uuid: "6a91d93684de436aa16f29de26a05a56" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34873 } } peers { permanent_uuid: "61b1ab2ca3d04f4c91bccf85b8513abf" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40947 } attrs { promote: false } } }
I20260809 07:56:15.689117 15498 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace6c8ef84a0498baf696b53b2ce87d4 [sys.catalog]: This master's current role is: LEADER
W20260809 07:56:15.961879 15708 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:15.962162 15708 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:15.962260 15708 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:56:15.962318 15708 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:15.970290 15708 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:15.970477 15708 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:15.980408 15708 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33531
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:15.981972 15708 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:15.983281 15708 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:15.992411 15720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:15.992650 15722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:15.994011 15708 server_base.cc:1061] running on GCE node
W20260809 07:56:15.994011 15725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:15.994712 15708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:15.995609 15708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:15.996909 15708 hybrid_clock.cc:648] HybridClock initialized: now 1786262175996844 us; error 31 us; skew 500 ppm
I20260809 07:56:15.997195 15708 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:15 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262175, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:16.000438 15708 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:16.002126 15708 webserver.cc:533] Webserver started at http://127.26.37.1:33531/ using document root <none> and password file <none>
I20260809 07:56:16.002800 15708 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:16.002884 15708 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:16.008231 15708 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 07:56:16.011552 15740 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:16.012835 15708 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 07:56:16.012962 15708 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "069c89cf557144c7a26850fd845c4668"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:16.013688 15708 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:16.031637 15708 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:16.035066 15708 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:16.035298 15708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:16.036264 15708 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:16.038018 15708 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:16.038116 15708 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:16.038233 15708 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:16.038318 15708 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:56:16.038692 15708 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:56:16.038753 15708 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:56:16.038877 15708 fault_injection.cc:59] injecting fault for kudu (pid 15708): FLAGS_fault_before_start (process will exit)
I20260809 07:56:16.064093 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:43689
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873
--webserver_port=33531
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:16.421217 15748 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:16.421603 15748 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:16.421761 15748 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:16.429787 15748 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:16.429993 15748 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:16.439795 15748 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:43689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33531
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41657,127.26.37.61:35281,127.26.37.60:34873,127.26.37.59:40947
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:16.441421 15748 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:16.442778 15748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:16.452226 15758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:16.452335 15759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:16.452904 15761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:16.453752 15748 server_base.cc:1061] running on GCE node
I20260809 07:56:16.454281 15748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:16.455134 15748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:16.456363 15748 hybrid_clock.cc:648] HybridClock initialized: now 1786262176456328 us; error 50 us; skew 500 ppm
I20260809 07:56:16.456696 15748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:16 dist-test-slave-bw88 krb5kdc[14269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262176, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:16.460192 15748 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:16.462080 15748 webserver.cc:533] Webserver started at http://127.26.37.1:33531/ using document root <none> and password file <none>
I20260809 07:56:16.462782 15748 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:16.462922 15748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:16.468206 15748 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 07:56:16.470959 15774 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:16.472229 15748 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:56:16.472398 15748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "069c89cf557144c7a26850fd845c4668"
format_stamp: "Formatted at 2026-08-09 07:56:12 on dist-test-slave-bw88"
I20260809 07:56:16.473048 15748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:16.494146 15748 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:16.498301 15748 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:16.498565 15748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:16.499554 15748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:16.501121 15748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:16.501228 15748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:16.501405 15748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:16.501482 15748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:16.535048 15748 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43689
I20260809 07:56:16.535095 15936 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43689 every 8 connection(s)
I20260809 07:56:16.536603 15748 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:16.538728 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15748
I20260809 07:56:16.539073 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15748
W20260809 07:56:16.547057 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15748: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:43689: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:16.547199 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15748
I20260809 07:56:16.575766 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15369
W20260809 07:56:16.576465 15751 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:16.541783 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:58437 (local address 127.26.37.62:41657)
0809 07:56:16.541974 (+ 191us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:16.541981 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:16.542262 (+ 281us) server_negotiation.cc:408] Connection header received
0809 07:56:16.545270 (+ 3008us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:16.545281 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:16.545424 (+ 143us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:16.545515 (+ 91us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:16.575631 (+ 30116us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:58437: BlockingRecv error: recv got EOF from 127.26.37.1:58437 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 07:56:16.576529 15948 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:16.541195 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:54435 (local address 127.26.37.61:35281)
0809 07:56:16.541728 (+ 533us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:16.541734 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:16.575508 (+ 33774us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:54435: BlockingRecv error: recv got EOF from 127.26.37.1:54435 (error 108)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":669,"threads_started":1}
W20260809 07:56:16.577195 15957 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:16.542760 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:47933 (local address 127.26.37.60:34873)
0809 07:56:16.557015 (+ 14255us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:16.557020 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:16.575650 (+ 18630us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:47933: BlockingRecv error: recv got EOF from 127.26.37.1:47933 (error 108)
Metrics: {"server-negotiator.queue_time_us":14149,"thread_start_us":14028,"threads_started":1}
W20260809 07:56:16.578467 15751 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:16.576668 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.26.37.62:41657)
0809 07:56:16.576880 (+ 212us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:16.576885 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:16.577006 (+ 121us) server_negotiation.cc:408] Connection header received
0809 07:56:16.577205 (+ 199us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:16.577210 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:16.577302 (+ 92us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:16.577448 (+ 146us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:16.578295 (+ 847us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42916: BlockingRecv error: recv got EOF from 127.0.0.1:42916 (error 108)
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":116}
W20260809 07:56:16.578426 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15369: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:41657: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:16.578603 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15369
I20260809 07:56:16.608381 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14400
I20260809 07:56:16.609598 14488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:16.756085 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14400
W20260809 07:56:16.784706 15067 connection.cc:573] server connection from 127.0.0.1:50204 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:56:16.785089 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14512
I20260809 07:56:16.786334 14598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:16.897756 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14512
I20260809 07:56:16.927527 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 14892
I20260809 07:56:16.928810 15129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:17.041949 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 14892
2026-08-09T07:56:17Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/7 (10291 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/8
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:56:17 dist-test-slave-bw88 krb5kdc[15964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:56:17 dist-test-slave-bw88 krb5kdc[15964](info): set up 2 sockets
Aug 09 07:56:17 dist-test-slave-bw88 krb5kdc[15964](info): commencing operation
krb5kdc: starting...
W20260809 07:56:19.294800 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.197s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:56:19 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:56:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:56:19Z Disabled control of system clock
2026-08-09T07:56:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:56:19Z Disabled control of system clock
2026-08-09T07:56:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:56:19Z Disabled control of system clock
2026-08-09T07:56:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:56:19Z Disabled control of system clock
2026-08-09T07:56:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:56:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:19.856835 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:42517
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:20.208309 15994 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:20.208599 15994 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:20.208695 15994 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:20.216514 15994 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:20.216609 15994 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:20.216674 15994 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:20.216732 15994 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:20.226950 15994 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:42517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:20.228395 15994 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:20.229874 15994 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:20.237876 16000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:20.237854 15999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:20.238498 16002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:20.239377 15994 server_base.cc:1061] running on GCE node
I20260809 07:56:20.240072 15994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:20.241678 15994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:20.242918 15994 hybrid_clock.cc:648] HybridClock initialized: now 1786262180242786 us; error 106 us; skew 500 ppm
I20260809 07:56:20.243251 15994 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:20 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:20.246277 15994 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:56:20.247817 15994 webserver.cc:533] Webserver started at http://127.26.37.62:45119/ using document root <none> and password file <none>
I20260809 07:56:20.248515 15994 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:20.248673 15994 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:20.249032 15994 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:20.251447 15994 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "59f72913f4084e69ba8b1e29ae82cccc"
format_stamp: "Formatted at 2026-08-09 07:56:20 on dist-test-slave-bw88"
I20260809 07:56:20.252213 15994 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "59f72913f4084e69ba8b1e29ae82cccc"
format_stamp: "Formatted at 2026-08-09 07:56:20 on dist-test-slave-bw88"
I20260809 07:56:20.257449 15994 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260809 07:56:20.260566 16012 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:20.261940 15994 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:20.262131 15994 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "59f72913f4084e69ba8b1e29ae82cccc"
format_stamp: "Formatted at 2026-08-09 07:56:20 on dist-test-slave-bw88"
I20260809 07:56:20.262326 15994 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:20.295377 15994 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:20.299506 15994 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:20.299873 15994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:20.322865 15994 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:42517
I20260809 07:56:20.322896 16087 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:42517 every 8 connection(s)
I20260809 07:56:20.324599 15994 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:56:20.328998 16088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:20.330476 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 15994
I20260809 07:56:20.330732 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:56:20.337673 16088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:20 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:20.364836 16088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:20.366052 16017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37219: connect: Connection refused (error 111)
W20260809 07:56:20.367818 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37219: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37219: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:20.417208 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } attempt: 1
W20260809 07:56:20.419395 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37219: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37219: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:20.478605 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37219
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:20.487668 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } attempt: 2
W20260809 07:56:20.489971 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37219: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37219: connect: Connection refused (error 111)
I20260809 07:56:20.581288 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } attempt: 3
W20260809 07:56:20.583590 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37219: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37219: connect: Connection refused (error 111)
I20260809 07:56:20.726846 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } attempt: 4
W20260809 07:56:20.729326 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:37219: Network error: Client connection negotiation failed: client connection to 127.26.37.61:37219: connect: Connection refused (error 111)
W20260809 07:56:20.853179 16107 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:20.853647 16107 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:20.853816 16107 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:20.862545 16107 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:20.862705 16107 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:20.862798 16107 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:20.862856 16107 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:20.875090 16107 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:37219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:20.876750 16107 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:20.878367 16107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:20.887171 16116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:20.887164 16115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:20.888610 16118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:20.888994 16107 server_base.cc:1061] running on GCE node
I20260809 07:56:20.889811 16107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:20.891469 16107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:20.892711 16107 hybrid_clock.cc:648] HybridClock initialized: now 1786262180892551 us; error 131 us; skew 500 ppm
I20260809 07:56:20.893061 16107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:20 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:20.896272 16107 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:56:20.898006 16107 webserver.cc:533] Webserver started at http://127.26.37.61:37323/ using document root <none> and password file <none>
I20260809 07:56:20.898712 16107 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:20.898855 16107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:20.899220 16107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:20.901816 16107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "4174d6fe98bc4822956e947671a08a99"
format_stamp: "Formatted at 2026-08-09 07:56:20 on dist-test-slave-bw88"
I20260809 07:56:20.902642 16107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "4174d6fe98bc4822956e947671a08a99"
format_stamp: "Formatted at 2026-08-09 07:56:20 on dist-test-slave-bw88"
I20260809 07:56:20.907866 16107 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:56:20.911303 16128 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:20.912588 16107 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:56:20.912776 16107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "4174d6fe98bc4822956e947671a08a99"
format_stamp: "Formatted at 2026-08-09 07:56:20 on dist-test-slave-bw88"
I20260809 07:56:20.912972 16107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:20.930552 16107 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:20.938473 16107 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:20.938740 16107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:20.959730 16107 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:37219
I20260809 07:56:20.959815 16202 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:37219 every 8 connection(s)
I20260809 07:56:20.961395 16107 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:56:20.965862 16203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:20.971701 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16107
I20260809 07:56:20.971913 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:56:20.972194 16203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:20 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:20.994094 16203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:20 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
I20260809 07:56:21.010679 16203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:21.011686 16129 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
W20260809 07:56:21.013458 16203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
I20260809 07:56:21.028623 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } attempt: 5
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:21.044035 16088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:21.046169 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:21.062816 16203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 1
W20260809 07:56:21.064879 16203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:56:21.097433 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 1
W20260809 07:56:21.099490 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:21.124413 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:34761
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:21.133104 16203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 2
W20260809 07:56:21.135210 16203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
I20260809 07:56:21.167757 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 2
W20260809 07:56:21.169978 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
I20260809 07:56:21.226536 16203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 3
W20260809 07:56:21.228744 16203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
I20260809 07:56:21.276201 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 3
W20260809 07:56:21.278420 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
I20260809 07:56:21.372006 16203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 4
W20260809 07:56:21.374264 16203 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
I20260809 07:56:21.455715 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 4
W20260809 07:56:21.457814 16088 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:34761: Network error: Client connection negotiation failed: client connection to 127.26.37.60:34761: connect: Connection refused (error 111)
W20260809 07:56:21.477881 16214 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:21.478235 16214 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:21.478389 16214 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:21.486333 16214 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:21.486480 16214 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:21.486601 16214 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:21.486694 16214 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:21.497009 16214 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:34761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:21.498930 16214 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:21.500366 16214 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:21.508503 16219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:21.508453 16220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:21.508948 16222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:21.510327 16214 server_base.cc:1061] running on GCE node
I20260809 07:56:21.511094 16214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:21.512588 16214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:21.513870 16214 hybrid_clock.cc:648] HybridClock initialized: now 1786262181513702 us; error 135 us; skew 500 ppm
I20260809 07:56:21.514250 16214 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262181, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:21.517328 16214 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:56:21.519105 16214 webserver.cc:533] Webserver started at http://127.26.37.60:40303/ using document root <none> and password file <none>
I20260809 07:56:21.519794 16214 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:21.519933 16214 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:21.520264 16214 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:21.522704 16214 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "3a57cf96ed7a4f9f9b5773da85adba1a"
format_stamp: "Formatted at 2026-08-09 07:56:21 on dist-test-slave-bw88"
I20260809 07:56:21.523499 16214 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "3a57cf96ed7a4f9f9b5773da85adba1a"
format_stamp: "Formatted at 2026-08-09 07:56:21 on dist-test-slave-bw88"
I20260809 07:56:21.528702 16214 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:56:21.532029 16231 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:21.533324 16214 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:56:21.533617 16214 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "3a57cf96ed7a4f9f9b5773da85adba1a"
format_stamp: "Formatted at 2026-08-09 07:56:21 on dist-test-slave-bw88"
I20260809 07:56:21.533818 16214 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:21.582839 16214 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:21.586583 16214 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:21.586885 16214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:21.607666 16214 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:34761
I20260809 07:56:21.607689 16302 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:34761 every 8 connection(s)
I20260809 07:56:21.609203 16214 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:56:21.613487 16304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:21.616366 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16214
I20260809 07:56:21.616571 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:56:21.620847 16304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262181, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:21.655161 16304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262181, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:21.669358 16304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:56:21.673491 16203 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 5
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262181, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:21.688150 16304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: Bootstrap starting.
I20260809 07:56:21.691293 16304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:21.692725 16304 log.cc:826] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:21.693043 16203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: Bootstrap starting.
I20260809 07:56:21.695747 16304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: No bootstrap required, opened a new log
I20260809 07:56:21.696403 16203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:21.697449 16203 log.cc:826] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:21.699936 16203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: No bootstrap required, opened a new log
I20260809 07:56:21.701205 16304 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.701543 16304 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:21.701689 16304 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a57cf96ed7a4f9f9b5773da85adba1a, State: Initialized, Role: FOLLOWER
I20260809 07:56:21.702255 16304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.703166 16316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.703510 16316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:21.704252 16304 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:21.705879 16203 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.706902 16203 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:21.707060 16203 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4174d6fe98bc4822956e947671a08a99, State: Initialized, Role: FOLLOWER
I20260809 07:56:21.707573 16203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.708559 16324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.708935 16324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:21.709784 16203 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:56:21.715394 16331 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:21.715595 16331 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:21.713393 16214 master_runner.cc:187] Error getting master registration for 127.26.37.62:42517: OK, instance_id { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" instance_seqno: 1786262180316920 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:56:21.723416 16214 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:56:21.729362 16353 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:21.729583 16353 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:21.726519 16107 master_runner.cc:187] Error getting master registration for 127.26.37.62:42517: OK, instance_id { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" instance_seqno: 1786262180316920 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:56:21.733032 16107 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:56:21.735009 16088 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } attempt: 5
Aug 09 07:56:21 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:21.752633 16088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: Bootstrap starting.
I20260809 07:56:21.755849 16088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:21.756953 16088 log.cc:826] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:21.759907 16088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: No bootstrap required, opened a new log
I20260809 07:56:21.766091 16088 raft_consensus.cc:359] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.766396 16088 raft_consensus.cc:385] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:21.766549 16088 raft_consensus.cc:740] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59f72913f4084e69ba8b1e29ae82cccc, State: Initialized, Role: FOLLOWER
I20260809 07:56:21.767256 16088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.768280 16356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.768646 16356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:21.769537 16088 sys_catalog.cc:565] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:21.785444 15994 master_runner.cc:264] No leader master found from master 59f72913f4084e69ba8b1e29ae82cccc
I20260809 07:56:21.785627 15994 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:56:21.797838 16371 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:21.798077 16371 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 07:56:21.803084 16019 tablet.cc:2406] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:56:21.839129 16238 tablet.cc:2406] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:56:21.847103 16324 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:56:21.847373 16324 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.850436 16053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4174d6fe98bc4822956e947671a08a99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f72913f4084e69ba8b1e29ae82cccc" is_pre_election: true
I20260809 07:56:21.851189 16053 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4174d6fe98bc4822956e947671a08a99 in term 0.
I20260809 07:56:21.851987 16136 leader_election.cc:304] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4174d6fe98bc4822956e947671a08a99, 59f72913f4084e69ba8b1e29ae82cccc; no voters:
I20260809 07:56:21.853406 16267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4174d6fe98bc4822956e947671a08a99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" is_pre_election: true
I20260809 07:56:21.853895 16267 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4174d6fe98bc4822956e947671a08a99 in term 0.
I20260809 07:56:21.849259 16324 leader_election.cc:290] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59f72913f4084e69ba8b1e29ae82cccc (127.26.37.62:42517), 3a57cf96ed7a4f9f9b5773da85adba1a (127.26.37.60:34761)
I20260809 07:56:21.854651 16324 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:56:21.854792 16324 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:56:21.854888 16324 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:21.863143 16324 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.867863 16267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4174d6fe98bc4822956e947671a08a99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a"
I20260809 07:56:21.868070 16267 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:21.871662 16267 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4174d6fe98bc4822956e947671a08a99 in term 1.
I20260809 07:56:21.872203 16129 leader_election.cc:304] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a57cf96ed7a4f9f9b5773da85adba1a, 4174d6fe98bc4822956e947671a08a99; no voters:
I20260809 07:56:21.872970 16324 leader_election.cc:290] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [CANDIDATE]: Term 1 election: Requested vote from peers 59f72913f4084e69ba8b1e29ae82cccc (127.26.37.62:42517), 3a57cf96ed7a4f9f9b5773da85adba1a (127.26.37.60:34761)
I20260809 07:56:21.877460 16324 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:56:21.877902 16324 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 1 LEADER]: Becoming Leader. State: Replica: 4174d6fe98bc4822956e947671a08a99, State: Running, Role: LEADER
I20260809 07:56:21.878664 16324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } }
I20260809 07:56:21.890733 16324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4174d6fe98bc4822956e947671a08a99. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.892335 16053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4174d6fe98bc4822956e947671a08a99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f72913f4084e69ba8b1e29ae82cccc"
I20260809 07:56:21.892549 16053 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:21.896157 16053 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4174d6fe98bc4822956e947671a08a99 in term 1.
I20260809 07:56:21.897085 16324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:21.904023 16378 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:56:21.909701 16378 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:56:21.924621 16267 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 1 FOLLOWER]: Refusing update from remote peer 4174d6fe98bc4822956e947671a08a99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:56:21.925720 16324 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:56:21.936048 16053 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 1 FOLLOWER]: Refusing update from remote peer 4174d6fe98bc4822956e947671a08a99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:56:21.937022 16324 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:56:21.941262 16138 tablet.cc:2406] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:56:21.962988 16356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4174d6fe98bc4822956e947671a08a99. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.963358 16356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:21.968120 16381 mvcc.cc:204] Tried to move back new op lower bound from 7316529897152667648 to 7316529896988045312. Current Snapshot: MvccSnapshot[applied={T|T < 7316529897152667648}]
I20260809 07:56:21.974722 16316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4174d6fe98bc4822956e947671a08a99. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.974992 16316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:21.976928 16380 mvcc.cc:204] Tried to move back new op lower bound from 7316529897152667648 to 7316529896988045312. Current Snapshot: MvccSnapshot[applied={T|T < 7316529897152667648}]
I20260809 07:56:21.979822 16356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.980023 16356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:21.990103 16316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.990291 16316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:21.994000 16324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.994258 16324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:21.994598 16324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } }
I20260809 07:56:21.994776 16324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:22.003988 16378 catalog_manager.cc:1383] Generated new cluster ID: 4d54f4d14321448990fb6a3499dc1006
I20260809 07:56:22.004094 16378 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:56:22.047044 16378 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:56:22.048693 16378 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:56:22.073089 16378 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: Generated new TSK 0
I20260809 07:56:22.074319 16378 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:22.267338 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:22.678692 16393 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:22.679013 16393 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:22.679111 16393 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:22.687031 16393 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:22.687193 16393 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:22.697208 16393 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:22.698827 16393 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:22.700196 16393 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:22.709789 16401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:22.709815 16403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:22.710402 16407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:22.710906 16393 server_base.cc:1061] running on GCE node
I20260809 07:56:22.711514 16393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:22.712675 16393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:22.713912 16393 hybrid_clock.cc:648] HybridClock initialized: now 1786262182713758 us; error 124 us; skew 500 ppm
I20260809 07:56:22.714223 16393 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:22 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262182, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:22.717517 16331 catalog_manager.cc:1295] Loaded cluster ID: 4d54f4d14321448990fb6a3499dc1006
I20260809 07:56:22.717548 16393 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:22.717624 16331 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: loading cluster ID for follower catalog manager: success
I20260809 07:56:22.719311 16393 webserver.cc:533] Webserver started at http://127.26.37.1:38965/ using document root <none> and password file <none>
I20260809 07:56:22.720022 16393 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:22.720162 16393 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:22.720517 16331 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: acquiring CA information for follower catalog manager: success
I20260809 07:56:22.720563 16393 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:22.722285 16331 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:56:22.723088 16393 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "557942fc082b4e24b650ab5be174998f"
format_stamp: "Formatted at 2026-08-09 07:56:22 on dist-test-slave-bw88"
I20260809 07:56:22.723842 16393 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "557942fc082b4e24b650ab5be174998f"
format_stamp: "Formatted at 2026-08-09 07:56:22 on dist-test-slave-bw88"
I20260809 07:56:22.729022 16393 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:56:22.732304 16414 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:22.733671 16393 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:56:22.733846 16393 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "557942fc082b4e24b650ab5be174998f"
format_stamp: "Formatted at 2026-08-09 07:56:22 on dist-test-slave-bw88"
I20260809 07:56:22.734035 16393 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:22.755429 16393 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:22.759342 16393 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:22.759644 16393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:22.760643 16393 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:22.762317 16393 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:22.762470 16393 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:22.762617 16393 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:22.762712 16393 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:22.798902 16393 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39049
I20260809 07:56:22.798962 16582 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39049 every 8 connection(s)
I20260809 07:56:22.800097 16371 catalog_manager.cc:1295] Loaded cluster ID: 4d54f4d14321448990fb6a3499dc1006
I20260809 07:56:22.800182 16371 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: loading cluster ID for follower catalog manager: success
I20260809 07:56:22.800482 16393 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:22.800832 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16393
I20260809 07:56:22.801052 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:56:22.803951 16371 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: acquiring CA information for follower catalog manager: success
I20260809 07:56:22.813267 16371 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:56:22 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262182, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:22 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262182, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:22.838461 16585 heartbeater.cc:344] Connected to a master server at 127.26.37.60:34761
I20260809 07:56:22.838894 16585 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:22.839073 16586 heartbeater.cc:344] Connected to a master server at 127.26.37.62:42517
I20260809 07:56:22.839392 16586 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:22.840132 16585 heartbeater.cc:507] Master 127.26.37.60:34761 requested a full tablet report, sending...
Aug 09 07:56:22 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262182, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:22.840391 16586 heartbeater.cc:507] Master 127.26.37.62:42517 requested a full tablet report, sending...
I20260809 07:56:22.843046 16254 ts_manager.cc:194] Registered new tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:22.843344 16035 ts_manager.cc:194] Registered new tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:22.848150 16591 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37219
I20260809 07:56:22.848304 16591 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:22.848762 16591 heartbeater.cc:507] Master 127.26.37.61:37219 requested a full tablet report, sending...
I20260809 07:56:22.850222 16152 ts_manager.cc:194] Registered new tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:22.851735 16152 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:34669
I20260809 07:56:22.858733 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:23.000273 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:46383
--webserver_interface=127.26.37.59
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:23.361125 16602 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:23.361533 16602 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:23.361631 16602 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:23.369757 16602 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:23.369848 16602 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:23.369905 16602 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:23.369952 16602 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:23.380645 16602 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:46383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:23.382400 16602 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:23.383786 16602 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:23.392297 16616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:23.392262 16617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:23.393608 16623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:23.394169 16602 server_base.cc:1061] running on GCE node
I20260809 07:56:23.394941 16602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:23.396602 16602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:23.397977 16602 hybrid_clock.cc:648] HybridClock initialized: now 1786262183397819 us; error 119 us; skew 500 ppm
I20260809 07:56:23.398407 16602 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262183, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:23.401886 16602 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:56:23.403492 16602 webserver.cc:533] Webserver started at http://127.26.37.59:46787/ using document root <none> and password file <none>
I20260809 07:56:23.404207 16602 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:23.404306 16602 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:23.404596 16602 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:23.407245 16602 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "f263b69fe5bd49c88e2e64084d1d7a63"
format_stamp: "Formatted at 2026-08-09 07:56:23 on dist-test-slave-bw88"
I20260809 07:56:23.408061 16602 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "f263b69fe5bd49c88e2e64084d1d7a63"
format_stamp: "Formatted at 2026-08-09 07:56:23 on dist-test-slave-bw88"
I20260809 07:56:23.413509 16602 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 07:56:23.416901 16634 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:23.418308 16602 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:23.418503 16602 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "f263b69fe5bd49c88e2e64084d1d7a63"
format_stamp: "Formatted at 2026-08-09 07:56:23 on dist-test-slave-bw88"
I20260809 07:56:23.418700 16602 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:23.449122 16602 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:23.452901 16602 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:23.453194 16602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:23.474794 16602 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:46383
I20260809 07:56:23.474828 16702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:46383 every 8 connection(s)
I20260809 07:56:23.476468 16602 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:56:23.480607 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:23.486104 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16602
I20260809 07:56:23.486361 16703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262183, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:23.507390 16703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262183, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:23.520623 16703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262183, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:23.532793 16703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262183, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:23.549264 16703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Bootstrap starting.
I20260809 07:56:23.552560 16703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:23.553757 16703 log.cc:826] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:23.556483 16703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: No bootstrap required, opened a new log
I20260809 07:56:23.562260 16703 raft_consensus.cc:359] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } }
I20260809 07:56:23.562587 16703 raft_consensus.cc:385] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:23.562697 16703 raft_consensus.cc:740] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f263b69fe5bd49c88e2e64084d1d7a63, State: Initialized, Role: FOLLOWER
I20260809 07:56:23.563333 16703 consensus_queue.cc:260] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } }
I20260809 07:56:23.564451 16713 sys_catalog.cc:455] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } } }
I20260809 07:56:23.564836 16713 sys_catalog.cc:458] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:23.565431 16703 sys_catalog.cc:565] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:56:23.575908 16730 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:23.576073 16730 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:23.584594 16602 master_runner.cc:429] Detected that this master f263b69fe5bd49c88e2e64084d1d7a63 is joining an existing cluster
I20260809 07:56:23.584734 16602 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:56:23.585850 16602 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:56:23.586208 16602 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:56:23.586400 16602 tablet_replica.cc:333] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: stopping tablet replica
I20260809 07:56:23.588006 16602 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:46383
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262180, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:23.660003 16152 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:46383: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 46383
}
attrs {
promote: true
}
}
dest_uuid: "4174d6fe98bc4822956e947671a08a99"
cas_config_opid_index: -1
I20260809 07:56:23.661418 16152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } }
I20260809 07:56:23.664218 16053 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 1 FOLLOWER]: Refusing update from remote peer 4174d6fe98bc4822956e947671a08a99: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:56:23.665350 16604 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 07:56:23.667635 16133 consensus_peers.cc:597] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 -> Peer f263b69fe5bd49c88e2e64084d1d7a63 (127.26.37.59:46383): Couldn't send request to peer f263b69fe5bd49c88e2e64084d1d7a63. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:56:23.670331 16266 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 1 FOLLOWER]: Refusing update from remote peer 4174d6fe98bc4822956e947671a08a99: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:56:23.669883 16604 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f263b69fe5bd49c88e2e64084d1d7a63 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:23.670614 16053 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f263b69fe5bd49c88e2e64084d1d7a63 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:23.674496 16748 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:56:23.681288 16604 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:46383
I20260809 07:56:23.682120 16602 master.cc:562] Master@127.26.37.59:46383 shutting down...
I20260809 07:56:23.682912 16604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:23.683234 16604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:23.683933 16755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:23.684353 16755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:23.689015 16266 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f263b69fe5bd49c88e2e64084d1d7a63 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:23.697794 16763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:23.698210 16763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:23.701570 16754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:23.701908 16754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:23.720631 16602 master.cc:584] Master@127.26.37.59:46383 shutdown complete.
I20260809 07:56:23.720728 16602 master_runner.cc:306] Clearing existing system tablet
I20260809 07:56:23.721166 16602 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:23.721309 16602 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:23.724051 16602 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 07:56:23.725767 16766 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:23.726183 16602 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:23.726455 16602 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 07:56:23.726559 16602 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "f263b69fe5bd49c88e2e64084d1d7a63"
format_stamp: "Formatted at 2026-08-09 07:56:23 on dist-test-slave-bw88"
I20260809 07:56:23.731156 16602 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:56:23.738212 16602 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:56:23.738394 16602 log.cc:1199] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:56:23.738896 16602 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Deleting consensus metadata
I20260809 07:56:23.739200 16602 master_runner.cc:316] Copying system tablet from 127.26.37.61:37219
I20260809 07:56:23.741833 16602 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.61:37219
I20260809 07:56:23.755494 16184 tablet_copy_service.cc:140] P 4174d6fe98bc4822956e947671a08a99: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f263b69fe5bd49c88e2e64084d1d7a63 ({username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:47870)
I20260809 07:56:23.755789 16184 tablet_copy_service.cc:161] P 4174d6fe98bc4822956e947671a08a99: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f263b69fe5bd49c88e2e64084d1d7a63 at {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:47870: session id = f263b69fe5bd49c88e2e64084d1d7a63-00000000000000000000000000000000
I20260809 07:56:23.760048 16184 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:56:23.762501 16602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:56:23.769568 16602 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: tablet copy: Starting download of 0 data blocks...
I20260809 07:56:23.769877 16602 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: tablet copy: Starting download of 1 WAL segments...
I20260809 07:56:23.772873 16602 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:56:23.777483 16184 tablet_copy_service.cc:342] P 4174d6fe98bc4822956e947671a08a99: Request end of tablet copy session f263b69fe5bd49c88e2e64084d1d7a63-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:47870
I20260809 07:56:23.777717 16184 tablet_copy_service.cc:434] P 4174d6fe98bc4822956e947671a08a99: ending tablet copy session f263b69fe5bd49c88e2e64084d1d7a63-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f263b69fe5bd49c88e2e64084d1d7a63
W20260809 07:56:23.785108 16602 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260809 07:56:23.788353 16602 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:23.790964 16776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:23.791462 16775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:23.797592 16602 server_base.cc:1061] running on GCE node
W20260809 07:56:23.797621 16782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:23.798169 16602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:23.799016 16602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:23.800271 16602 hybrid_clock.cc:648] HybridClock initialized: now 1786262183800150 us; error 99 us; skew 500 ppm
I20260809 07:56:23.800510 16602 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262183, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:23.802556 16602 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:56:23.806310 16602 webserver.cc:533] Webserver started at http://127.26.37.59:45137/ using document root <none> and password file <none>
I20260809 07:56:23.806686 16602 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:23.806821 16602 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:23.809284 16602 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:56:23.811036 16788 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:23.811576 16602 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 07:56:23.811721 16602 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "f263b69fe5bd49c88e2e64084d1d7a63"
format_stamp: "Formatted at 2026-08-09 07:56:23 on dist-test-slave-bw88"
I20260809 07:56:23.811889 16602 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:23.821895 16602 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:23.825070 16602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:23.845098 16602 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:46383
I20260809 07:56:23.845129 16858 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:46383 every 8 connection(s)
I20260809 07:56:23.846267 16602 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:56:23.851315 16859 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:56:23.853683 16859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Bootstrap starting.
I20260809 07:56:23.854357 16591 heartbeater.cc:499] Master 127.26.37.61:37219 was elected leader, sending a full tablet report...
I20260809 07:56:23.867731 16859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:56:23.868204 16859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: Bootstrap complete.
I20260809 07:56:23.869119 16859 raft_consensus.cc:359] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } }
I20260809 07:56:23.869305 16859 raft_consensus.cc:740] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f263b69fe5bd49c88e2e64084d1d7a63, State: Initialized, Role: LEARNER
I20260809 07:56:23.869620 16859 consensus_queue.cc:260] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } }
I20260809 07:56:23.870447 16862 sys_catalog.cc:455] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:23.870738 16859 sys_catalog.cc:565] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:23.870785 16862 sys_catalog.cc:458] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:56:23.880442 16878 catalog_manager.cc:1295] Loaded cluster ID: 4d54f4d14321448990fb6a3499dc1006
I20260809 07:56:23.880564 16878 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: loading cluster ID for follower catalog manager: success
I20260809 07:56:23.889730 16878 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: acquiring CA information for follower catalog manager: success
I20260809 07:56:23.892510 16878 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:56:23.894030 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16393
Aug 09 07:56:23 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:56:23.914753 16576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:24.015651 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16393
I20260809 07:56:24.046020 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39049
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--webserver_port=38965
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:24.208776 16748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:24.209192 16748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:24.291513 16824 raft_consensus.cc:1217] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:56:24.292371 16862 sys_catalog.cc:455] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4174d6fe98bc4822956e947671a08a99. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:24.292693 16862 sys_catalog.cc:458] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: This master's current role is: LEARNER
W20260809 07:56:24.418920 16882 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:24.419242 16882 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:24.419425 16882 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:24.427850 16882 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:24.428088 16882 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:24.438710 16882 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38965
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:24.440160 16882 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:24.441685 16882 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:24.451443 16893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:24.451524 16894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:24.452178 16898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:24.452772 16882 server_base.cc:1061] running on GCE node
I20260809 07:56:24.453292 16882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:24.454350 16882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:24.455566 16882 hybrid_clock.cc:648] HybridClock initialized: now 1786262184455444 us; error 91 us; skew 500 ppm
I20260809 07:56:24.455873 16882 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:24 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262184, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:24.459263 16882 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:24.460850 16882 webserver.cc:533] Webserver started at http://127.26.37.1:38965/ using document root <none> and password file <none>
I20260809 07:56:24.461694 16882 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:24.461845 16882 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:24.467106 16882 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:56:24.469874 16907 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:24.471251 16882 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:56:24.471410 16882 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "557942fc082b4e24b650ab5be174998f"
format_stamp: "Formatted at 2026-08-09 07:56:22 on dist-test-slave-bw88"
I20260809 07:56:24.472221 16882 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:24.484388 16882 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:24.488140 16882 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:24.488410 16882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:24.489354 16882 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:24.490917 16882 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:24.491040 16882 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:24.491137 16882 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:24.491209 16882 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:24.526973 16882 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39049
I20260809 07:56:24.527060 17072 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39049 every 8 connection(s)
I20260809 07:56:24.528568 16882 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:24.533617 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 16882
Aug 09 07:56:24 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262184, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:24 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262184, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:24 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262184, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:56:24 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262184, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:24.567589 17078 heartbeater.cc:344] Connected to a master server at 127.26.37.62:42517
I20260809 07:56:24.568075 17078 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:24.569213 17078 heartbeater.cc:507] Master 127.26.37.62:42517 requested a full tablet report, sending...
I20260809 07:56:24.569243 17077 heartbeater.cc:344] Connected to a master server at 127.26.37.60:34761
I20260809 07:56:24.569427 17077 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:24.569891 17077 heartbeater.cc:507] Master 127.26.37.60:34761 requested a full tablet report, sending...
I20260809 07:56:24.571343 16035 ts_manager.cc:194] Re-registered known tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:24.571563 16253 ts_manager.cc:194] Re-registered known tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:24.575543 17075 heartbeater.cc:344] Connected to a master server at 127.26.37.59:46383
I20260809 07:56:24.575676 17075 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:24.576182 17075 heartbeater.cc:507] Master 127.26.37.59:46383 requested a full tablet report, sending...
I20260809 07:56:24.577947 16809 ts_manager.cc:194] Registered new tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:24.578507 17079 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37219
I20260809 07:56:24.578641 17079 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:24.578972 17079 heartbeater.cc:507] Master 127.26.37.61:37219 requested a full tablet report, sending...
I20260809 07:56:24.579991 16152 ts_manager.cc:194] Re-registered known tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:24.580857 16152 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:54495
I20260809 07:56:24.588171 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:24.595598 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:24.595741 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 15994
I20260809 07:56:24.596516 16082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:24.714560 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 15994
I20260809 07:56:24.746709 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:42517
--webserver_interface=127.26.37.62
--webserver_port=45119
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:24.790725 16884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:24.791088 16884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:24.792639 17100 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99: attempting to promote NON_VOTER f263b69fe5bd49c88e2e64084d1d7a63 to VOTER
I20260809 07:56:24.793634 17100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } }
I20260809 07:56:24.796548 16824 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 1 LEARNER]: Refusing update from remote peer 4174d6fe98bc4822956e947671a08a99: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:24.797500 16266 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 1 FOLLOWER]: Refusing update from remote peer 4174d6fe98bc4822956e947671a08a99: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:24.798642 17101 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:24.799268 17101 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 07:56:24.807466 16136 consensus_peers.cc:597] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 -> Peer 59f72913f4084e69ba8b1e29ae82cccc (127.26.37.62:42517): Couldn't send request to peer 59f72913f4084e69ba8b1e29ae82cccc. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:42517: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:56:24.808513 17100 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f263b69fe5bd49c88e2e64084d1d7a63 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:24.809221 16266 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f263b69fe5bd49c88e2e64084d1d7a63 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:24.812491 17106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:24.812816 17100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:24.813182 17100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:24.813127 16884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:24.812829 17106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:24.813321 16884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:24.814095 16824 raft_consensus.cc:2955] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f263b69fe5bd49c88e2e64084d1d7a63 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:24.814944 17111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:24.815244 17111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a57cf96ed7a4f9f9b5773da85adba1a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:24.818284 17112 sys_catalog.cc:455] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:24.818593 17112 sys_catalog.cc:458] T 00000000000000000000000000000000 P f263b69fe5bd49c88e2e64084d1d7a63 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:56:25.127028 17099 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:25.127390 17099 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:25.127560 17099 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:25.135926 17099 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:25.136077 17099 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:25.136215 17099 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:25.136303 17099 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:25.147082 17099 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:42517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=45119
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:25.148695 17099 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:25.150102 17099 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:25.158123 17120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:25.158807 17119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:25.159336 17124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:25.159752 17099 server_base.cc:1061] running on GCE node
I20260809 07:56:25.160554 17099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:25.162197 17099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:25.163434 17099 hybrid_clock.cc:648] HybridClock initialized: now 1786262185163294 us; error 109 us; skew 500 ppm
I20260809 07:56:25.163784 17099 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:25 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262185, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:25.167512 17099 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:56:25.168952 17099 webserver.cc:533] Webserver started at http://127.26.37.62:45119/ using document root <none> and password file <none>
I20260809 07:56:25.169690 17099 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:25.169799 17099 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:25.175174 17099 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:56:25.178022 17136 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:25.179399 17099 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:25.179589 17099 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "59f72913f4084e69ba8b1e29ae82cccc"
format_stamp: "Formatted at 2026-08-09 07:56:20 on dist-test-slave-bw88"
I20260809 07:56:25.180302 17099 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:25.198477 17099 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:25.202003 17099 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:25.202272 17099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:25.223313 17099 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:42517
I20260809 07:56:25.223352 17212 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:42517 every 8 connection(s)
I20260809 07:56:25.225085 17099 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:56:25.231907 17213 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:56:25.231971 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 17099
I20260809 07:56:25.232321 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16882
I20260809 07:56:25.236584 17213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: Bootstrap starting.
I20260809 07:56:25.251708 17067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:25.258528 17213 log.cc:826] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:25.268600 17213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:56:25.269208 17213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: Bootstrap complete.
I20260809 07:56:25.276575 17213 raft_consensus.cc:359] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } }
I20260809 07:56:25.277312 17213 raft_consensus.cc:740] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59f72913f4084e69ba8b1e29ae82cccc, State: Initialized, Role: FOLLOWER
I20260809 07:56:25.278278 17213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } }
I20260809 07:56:25.279244 17220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: true } } }
I20260809 07:56:25.279632 17220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:25.280438 17213 sys_catalog.cc:565] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:25.293632 17237 catalog_manager.cc:1295] Loaded cluster ID: 4d54f4d14321448990fb6a3499dc1006
I20260809 07:56:25.293758 17237 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: loading cluster ID for follower catalog manager: success
I20260809 07:56:25.297937 17237 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: acquiring CA information for follower catalog manager: success
I20260809 07:56:25.300400 17237 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:56:25.323210 17172 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f263b69fe5bd49c88e2e64084d1d7a63 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:25.346848 17247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:25.347208 17247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:25.347898 17220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4174d6fe98bc4822956e947671a08a99. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:25.348138 17220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f72913f4084e69ba8b1e29ae82cccc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:25.374125 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16882
I20260809 07:56:25.404093 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39049
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--webserver_port=38965
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:25.776945 17250 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:25.777305 17250 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:25.777486 17250 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:25.786023 17250 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:25.786254 17250 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:25.796553 17250 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38965
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:25.798203 17250 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:25.799671 17250 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:25.809470 17255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:25.809449 17256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:25.810211 17260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:25.811098 17250 server_base.cc:1061] running on GCE node
I20260809 07:56:25.811642 17250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:25.812767 17250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:25.816164 17250 hybrid_clock.cc:648] HybridClock initialized: now 1786262185815948 us; error 159 us; skew 500 ppm
I20260809 07:56:25.816562 17250 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:25 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262185, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:25.820250 17250 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:25.821957 17250 webserver.cc:533] Webserver started at http://127.26.37.1:38965/ using document root <none> and password file <none>
I20260809 07:56:25.822680 17250 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:25.822811 17250 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:25.828234 17250 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 07:56:25.831048 17269 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:25.832412 17250 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:56:25.832599 17250 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "557942fc082b4e24b650ab5be174998f"
format_stamp: "Formatted at 2026-08-09 07:56:22 on dist-test-slave-bw88"
I20260809 07:56:25.833315 17250 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:25.874682 17250 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:25.878800 17250 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:25.879068 17250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:25.879977 17250 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:25.881695 17250 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:25.881791 17250 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:25.881893 17250 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:25.881976 17250 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:25.885144 16884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4174d6fe98bc4822956e947671a08a99" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "59f72913f4084e69ba8b1e29ae82cccc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42517 } } peers { permanent_uuid: "4174d6fe98bc4822956e947671a08a99" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 37219 } } peers { permanent_uuid: "3a57cf96ed7a4f9f9b5773da85adba1a" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 34761 } } peers { permanent_uuid: "f263b69fe5bd49c88e2e64084d1d7a63" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 46383 } attrs { promote: false } } }
I20260809 07:56:25.885509 16884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4174d6fe98bc4822956e947671a08a99 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:25.919842 17250 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39049
I20260809 07:56:25.919934 17433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39049 every 8 connection(s)
I20260809 07:56:25.921482 17250 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:25.929970 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 17250
Aug 09 07:56:25 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262185, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:25 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262185, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:56:25 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262185, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:25 dist-test-slave-bw88 krb5kdc[15964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262185, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:26.002431 17441 heartbeater.cc:344] Connected to a master server at 127.26.37.61:37219
I20260809 07:56:26.002893 17441 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:26.004043 17441 heartbeater.cc:507] Master 127.26.37.61:37219 requested a full tablet report, sending...
I20260809 07:56:26.006206 17438 heartbeater.cc:344] Connected to a master server at 127.26.37.60:34761
I20260809 07:56:26.006353 17438 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:26.006345 16152 ts_manager.cc:194] Re-registered known tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:26.006932 17438 heartbeater.cc:507] Master 127.26.37.60:34761 requested a full tablet report, sending...
I20260809 07:56:26.007455 16152 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:47529
I20260809 07:56:26.008494 16253 ts_manager.cc:194] Re-registered known tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:26.008648 17437 heartbeater.cc:344] Connected to a master server at 127.26.37.59:46383
I20260809 07:56:26.008759 17437 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:26.009256 17437 heartbeater.cc:507] Master 127.26.37.59:46383 requested a full tablet report, sending...
I20260809 07:56:26.010135 17440 heartbeater.cc:344] Connected to a master server at 127.26.37.62:42517
I20260809 07:56:26.010285 17440 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:26.010788 17440 heartbeater.cc:507] Master 127.26.37.62:42517 requested a full tablet report, sending...
I20260809 07:56:26.011713 16809 ts_manager.cc:194] Re-registered known tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
I20260809 07:56:26.012640 17159 ts_manager.cc:194] Registered new tserver with Master: 557942fc082b4e24b650ab5be174998f (127.26.37.1:39049)
W20260809 07:56:26.036862 26772 mini_kdc.cc:336] Time spent kdestroy: real 0.106s user 0.000s sys 0.036s
I20260809 07:56:26.040441 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17250
W20260809 07:56:26.041862 17467 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:26.038194 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:53700 (local address 127.26.37.1:39049)
0809 07:56:26.039005 (+ 811us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:26.039020 (+ 15us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:26.039056 (+ 36us) server_negotiation.cc:408] Connection header received
0809 07:56:26.039128 (+ 72us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:26.039135 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:26.039342 (+ 207us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:26.039508 (+ 166us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:26.040350 (+ 842us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53700: BlockingRecv error: recv got EOF from 127.0.0.1:53700 (error 108)
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":455,"threads_started":1}
W20260809 07:56:26.042788 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17250: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:39049: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:26.042939 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 17250
I20260809 07:56:26.068562 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39049
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--webserver_port=38965
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:26.427089 17470 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:26.427397 17470 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:26.427541 17470 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:56:26.427640 17470 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:26.435847 17470 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:26.436124 17470 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:26.446527 17470 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38965
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:26.448092 17470 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:26.449625 17470 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:26.459523 17478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:26.459520 17479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:26.459867 17481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:26.460453 17470 server_base.cc:1061] running on GCE node
I20260809 07:56:26.460964 17470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:26.462081 17470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:26.463392 17470 hybrid_clock.cc:648] HybridClock initialized: now 1786262186463208 us; error 138 us; skew 500 ppm
I20260809 07:56:26.463688 17470 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:26 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262186, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:26.466914 17470 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:26.468489 17470 webserver.cc:533] Webserver started at http://127.26.37.1:38965/ using document root <none> and password file <none>
I20260809 07:56:26.469198 17470 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:26.469336 17470 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:26.474568 17470 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:56:26.477173 17490 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:26.478515 17470 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:26.478685 17470 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "557942fc082b4e24b650ab5be174998f"
format_stamp: "Formatted at 2026-08-09 07:56:22 on dist-test-slave-bw88"
I20260809 07:56:26.479372 17470 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:26.506209 17470 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:26.510160 17470 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:26.510459 17470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:26.511399 17470 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:26.513149 17470 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:26.513247 17470 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:26.513408 17470 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:26.513497 17470 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:56:26.513891 17470 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:56:26.513985 17470 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:56:26.514066 17470 fault_injection.cc:59] injecting fault for kudu (pid 17470): FLAGS_fault_before_start (process will exit)
I20260809 07:56:26.534297 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:39049
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761
--webserver_port=38965
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:26.931049 17500 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:26.931407 17500 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:26.931568 17500 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:26.939766 17500 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:26.939994 17500 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:26.950193 17500 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39601,127.26.37.21:35979,127.26.37.22:42221,127.26.37.23:38953,127.26.37.24:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:39049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=38965
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:42517,127.26.37.61:37219,127.26.37.60:34761,127.26.37.59:46383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:26.951797 17500 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:26.953188 17500 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:26.963395 17509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:26.964772 17508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:26.965747 17511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:26.966040 17500 server_base.cc:1061] running on GCE node
I20260809 07:56:26.966872 17500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:56:26.968248 17500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:56:26.969501 17500 hybrid_clock.cc:648] HybridClock initialized: now 1786262186969356 us; error 109 us; skew 500 ppm
I20260809 07:56:26.969838 17500 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:26 dist-test-slave-bw88 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262186, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:26.973910 17500 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:26.975520 17500 webserver.cc:533] Webserver started at http://127.26.37.1:38965/ using document root <none> and password file <none>
I20260809 07:56:26.976395 17500 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:26.976541 17500 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:26.981971 17500 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:26.984867 17521 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:26.986363 17500 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:26.986545 17500 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "557942fc082b4e24b650ab5be174998f"
format_stamp: "Formatted at 2026-08-09 07:56:22 on dist-test-slave-bw88"
I20260809 07:56:26.987284 17500 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:27.010231 17500 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:27.014636 17500 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:27.014914 17500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:27.015880 17500 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:27.017570 17500 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:27.017706 17500 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:27.017835 17500 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:27.017930 17500 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:27.052688 17500 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39049
I20260809 07:56:27.052779 17676 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39049 every 8 connection(s)
I20260809 07:56:27.054322 17500 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:27.061864 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 17500
I20260809 07:56:27.062244 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17500
W20260809 07:56:27.080127 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17500: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:39049: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:27.080269 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 17500
W20260809 07:56:27.110170 17502 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:27.058211 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:58249 (local address 127.26.37.62:42517)
0809 07:56:27.058445 (+ 234us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:27.058454 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:27.065057 (+ 6603us) server_negotiation.cc:408] Connection header received
0809 07:56:27.068603 (+ 3546us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:27.068614 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:27.068733 (+ 119us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:27.068846 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:27.072180 (+ 3334us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:27.073921 (+ 1741us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:56:27.076076 (+ 2155us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:56:27.076742 (+ 666us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0809 07:56:27.109382 (+ 32640us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:58249: BlockingRecv error: recv got EOF from 127.26.37.1:58249 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 07:56:27.110193 17684 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:27.057821 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:56329 (local address 127.26.37.59:46383)
0809 07:56:27.059287 (+ 1466us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:27.059292 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:27.077966 (+ 18674us) server_negotiation.cc:408] Connection header received
0809 07:56:27.088418 (+ 10452us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:27.088426 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:27.088531 (+ 105us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:27.088642 (+ 111us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:27.109252 (+ 20610us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:56329: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":1374,"thread_start_us":1261,"threads_started":1}
W20260809 07:56:27.110217 17695 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:27.067542 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:41189 (local address 127.26.37.61:37219)
0809 07:56:27.089794 (+ 22252us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:27.089803 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:27.109189 (+ 19386us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:41189: BlockingRecv error: recv got EOF from 127.26.37.1:41189 (error 108)
Metrics: {"server-negotiator.queue_time_us":22107,"thread_start_us":21997,"threads_started":1}
W20260809 07:56:27.110237 17686 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:27.057825 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:40659 (local address 127.26.37.60:34761)
0809 07:56:27.058915 (+ 1090us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:27.058923 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:27.072430 (+ 13507us) server_negotiation.cc:408] Connection header received
0809 07:56:27.087691 (+ 15261us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:27.087702 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:27.087800 (+ 98us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:27.087913 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:27.109283 (+ 21370us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:40659: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":793,"threads_started":1}
I20260809 07:56:27.110996 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17099
W20260809 07:56:27.113124 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 17099: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:42517: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:27.113276 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 17099
I20260809 07:56:27.144604 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16107
I20260809 07:56:27.145840 16196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:27.260960 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16107
I20260809 07:56:27.294107 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16214
I20260809 07:56:27.295356 16296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:27.408300 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16214
I20260809 07:56:27.438194 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 16602
I20260809 07:56:27.439504 16853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:27.553637 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 16602
2026-08-09T07:56:27Z chronyd exiting
2026-08-09T07:56:27Z chronyd exiting
2026-08-09T07:56:27Z chronyd exiting
2026-08-09T07:56:27Z chronyd exiting
2026-08-09T07:56:27Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/8 (10553 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/9
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:56:27 dist-test-slave-bw88 krb5kdc[17702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:56:27 dist-test-slave-bw88 krb5kdc[17702](info): set up 2 sockets
Aug 09 07:56:27 dist-test-slave-bw88 krb5kdc[17702](info): commencing operation
krb5kdc: starting...
W20260809 07:56:29.854519 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.204s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:56:30 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:56:37: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:56:39 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262199, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:56:45 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262205, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:56:45 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262205, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:46.521440 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36135
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:46.878537 18141 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:46.878867 18141 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:46.879047 18141 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:46.887382 18141 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:46.887470 18141 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:46.887565 18141 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:46.887651 18141 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:46.897924 18141 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:46.899515 18141 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:46.900857 18141 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:46.909925 18149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:46.909895 18150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:46.910516 18152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:46.910790 18141 server_base.cc:1061] running on GCE node
I20260809 07:56:46.911554 18141 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:46.911725 18141 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:46.911792 18141 hybrid_clock.cc:648] HybridClock initialized: now 1786262206911774 us; error 0 us; skew 500 ppm
I20260809 07:56:46.912089 18141 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:46 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262206, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:46.915612 18141 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:56:46.917492 18141 webserver.cc:533] Webserver started at http://127.26.37.62:34781/ using document root <none> and password file <none>
I20260809 07:56:46.918149 18141 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:46.918242 18141 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:46.918555 18141 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:46.920861 18141 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "f0f3a3af3f474a77a91a7149b5c75150"
format_stamp: "Formatted at 2026-08-09 07:56:46 on dist-test-slave-bw88"
I20260809 07:56:46.921663 18141 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "f0f3a3af3f474a77a91a7149b5c75150"
format_stamp: "Formatted at 2026-08-09 07:56:46 on dist-test-slave-bw88"
I20260809 07:56:46.926687 18141 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 07:56:46.929927 18159 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:46.931165 18141 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:46.931340 18141 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "f0f3a3af3f474a77a91a7149b5c75150"
format_stamp: "Formatted at 2026-08-09 07:56:46 on dist-test-slave-bw88"
I20260809 07:56:46.931540 18141 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:46.947582 18141 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:46.960613 18141 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:46.960915 18141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:46.982283 18141 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36135
I20260809 07:56:46.982443 18228 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36135 every 8 connection(s)
I20260809 07:56:46.984073 18141 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:56:46.984757 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18141
I20260809 07:56:46.985073 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:56:46.988534 18229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:46.994225 18229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:47 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262206, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:47.017541 18229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:47.018716 18165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38325: connect: Connection refused (error 111)
W20260809 07:56:47.020614 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38325: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38325: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:47.069970 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } attempt: 1
W20260809 07:56:47.072099 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38325: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38325: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:47.131650 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38325
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:47.140347 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } attempt: 2
W20260809 07:56:47.142591 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38325: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38325: connect: Connection refused (error 111)
I20260809 07:56:47.233850 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } attempt: 3
W20260809 07:56:47.236294 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38325: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38325: connect: Connection refused (error 111)
I20260809 07:56:47.379577 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } attempt: 4
W20260809 07:56:47.381847 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:38325: Network error: Client connection negotiation failed: client connection to 127.26.37.61:38325: connect: Connection refused (error 111)
W20260809 07:56:47.514055 18243 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:47.514434 18243 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:47.514587 18243 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:47.522800 18243 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:47.522924 18243 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:47.523041 18243 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:47.523121 18243 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:47.533062 18243 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:38325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:47.534727 18243 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:47.536206 18243 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:47.544433 18252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:47.544480 18253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:47.545773 18243 server_base.cc:1061] running on GCE node
W20260809 07:56:47.546068 18258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:47.546905 18243 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:47.547056 18243 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:47.547145 18243 hybrid_clock.cc:648] HybridClock initialized: now 1786262207547103 us; error 0 us; skew 500 ppm
I20260809 07:56:47.547538 18243 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:47 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262207, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:47.551059 18243 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:56:47.553086 18243 webserver.cc:533] Webserver started at http://127.26.37.61:45149/ using document root <none> and password file <none>
I20260809 07:56:47.553879 18243 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:47.554028 18243 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:47.554373 18243 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:47.556768 18243 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "aee2b5750523447490519431cd052c33"
format_stamp: "Formatted at 2026-08-09 07:56:47 on dist-test-slave-bw88"
I20260809 07:56:47.557601 18243 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "aee2b5750523447490519431cd052c33"
format_stamp: "Formatted at 2026-08-09 07:56:47 on dist-test-slave-bw88"
I20260809 07:56:47.563009 18243 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:56:47.566315 18265 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:47.567636 18243 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:56:47.567828 18243 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "aee2b5750523447490519431cd052c33"
format_stamp: "Formatted at 2026-08-09 07:56:47 on dist-test-slave-bw88"
I20260809 07:56:47.568024 18243 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:47.579335 18243 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:47.583395 18243 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:47.583694 18243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:47.604532 18243 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:38325
I20260809 07:56:47.604550 18334 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:38325 every 8 connection(s)
I20260809 07:56:47.606604 18243 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:56:47.608404 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18243
I20260809 07:56:47.608624 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:56:47.611876 18335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:47.620644 18335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:47 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262207, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:47.643675 18335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
Aug 09 07:56:47 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262207, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:47.661521 18335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:47.662783 18266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
W20260809 07:56:47.664450 18335 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:56:47.681074 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } attempt: 5
Aug 09 07:56:47 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262206, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:47.696023 18229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:56:47.698089 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:56:47.713807 18335 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 1
W20260809 07:56:47.715756 18335 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:56:47.749313 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 1
W20260809 07:56:47.751483 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:47.762827 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:46289
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:47.783964 18335 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 2
W20260809 07:56:47.786192 18335 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:56:47.819754 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 2
W20260809 07:56:47.822126 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:56:47.877408 18335 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 3
W20260809 07:56:47.879714 18335 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:56:47.928354 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 3
W20260809 07:56:47.930644 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:56:48.022927 18335 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 4
W20260809 07:56:48.025053 18335 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
I20260809 07:56:48.107877 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 4
W20260809 07:56:48.110180 18229 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:46289: Network error: Client connection negotiation failed: client connection to 127.26.37.60:46289: connect: Connection refused (error 111)
W20260809 07:56:48.129321 18352 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:48.129747 18352 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:48.129911 18352 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:48.138360 18352 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:48.138507 18352 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:48.138626 18352 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:48.138710 18352 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:48.149781 18352 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:46289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:48.151816 18352 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:48.153296 18352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:48.161461 18359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:48.161468 18358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:48.162339 18362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:48.163174 18352 server_base.cc:1061] running on GCE node
I20260809 07:56:48.163915 18352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:48.164125 18352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:48.164227 18352 hybrid_clock.cc:648] HybridClock initialized: now 1786262208164188 us; error 0 us; skew 500 ppm
I20260809 07:56:48.164583 18352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262208, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:48.168177 18352 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:56:48.170066 18352 webserver.cc:533] Webserver started at http://127.26.37.60:40455/ using document root <none> and password file <none>
I20260809 07:56:48.170770 18352 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:48.170873 18352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:48.171175 18352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:48.173687 18352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "d3b24f399b0147b0bcb76cb4416063e8"
format_stamp: "Formatted at 2026-08-09 07:56:48 on dist-test-slave-bw88"
I20260809 07:56:48.174410 18352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "d3b24f399b0147b0bcb76cb4416063e8"
format_stamp: "Formatted at 2026-08-09 07:56:48 on dist-test-slave-bw88"
I20260809 07:56:48.179807 18352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 07:56:48.183130 18369 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:48.184561 18352 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:56:48.184746 18352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "d3b24f399b0147b0bcb76cb4416063e8"
format_stamp: "Formatted at 2026-08-09 07:56:48 on dist-test-slave-bw88"
I20260809 07:56:48.184950 18352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:48.209674 18352 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:48.213831 18352 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:48.214146 18352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:48.235478 18352 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:46289
I20260809 07:56:48.235529 18435 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:46289 every 8 connection(s)
I20260809 07:56:48.237169 18352 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:56:48.238289 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18352
I20260809 07:56:48.238498 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:56:48.242709 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:48.251396 18436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262208, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:48.273002 18436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262208, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:48.287137 18436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262208, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:48.304939 18436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: Bootstrap starting.
I20260809 07:56:48.308139 18436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:48.309166 18436 log.cc:826] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:48.312045 18436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: No bootstrap required, opened a new log
I20260809 07:56:48.317126 18436 raft_consensus.cc:359] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.317461 18436 raft_consensus.cc:385] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:48.317571 18436 raft_consensus.cc:740] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b24f399b0147b0bcb76cb4416063e8, State: Initialized, Role: FOLLOWER
I20260809 07:56:48.318105 18436 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.319088 18454 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.319473 18454 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:48.320243 18436 sys_catalog.cc:565] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:48.324276 18335 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 5
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262207, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
W20260809 07:56:48.337111 18471 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:48.337299 18471 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:48.329469 18352 master_runner.cc:187] Error getting master registration for 127.26.37.62:36135: OK, instance_id { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" instance_seqno: 1786262206975940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:56:48.337764 18352 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:56:48.344762 18335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33: Bootstrap starting.
I20260809 07:56:48.348604 18335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:48.349776 18335 log.cc:826] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:48.352331 18335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33: No bootstrap required, opened a new log
I20260809 07:56:48.358757 18335 raft_consensus.cc:359] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.359038 18335 raft_consensus.cc:385] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:48.359182 18335 raft_consensus.cc:740] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aee2b5750523447490519431cd052c33, State: Initialized, Role: FOLLOWER
I20260809 07:56:48.360174 18335 consensus_queue.cc:260] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.360517 18476 raft_consensus.cc:493] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:56:48.360873 18476 raft_consensus.cc:515] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.361843 18477 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.362246 18477 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:48.362515 18335 sys_catalog.cc:565] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:48.362875 18476 leader_election.cc:290] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0f3a3af3f474a77a91a7149b5c75150 (127.26.37.62:36135), d3b24f399b0147b0bcb76cb4416063e8 (127.26.37.60:46289)
I20260809 07:56:48.363763 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aee2b5750523447490519431cd052c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3b24f399b0147b0bcb76cb4416063e8" is_pre_election: true
I20260809 07:56:48.364339 18401 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aee2b5750523447490519431cd052c33 in term 0.
I20260809 07:56:48.365053 18266 leader_election.cc:304] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aee2b5750523447490519431cd052c33, d3b24f399b0147b0bcb76cb4416063e8; no voters:
I20260809 07:56:48.365464 18476 raft_consensus.cc:2804] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:56:48.365660 18476 raft_consensus.cc:493] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:56:48.365790 18476 raft_consensus.cc:3060] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:48.370322 18476 raft_consensus.cc:515] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.372813 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aee2b5750523447490519431cd052c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3b24f399b0147b0bcb76cb4416063e8"
I20260809 07:56:48.373133 18401 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:48.377979 18401 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aee2b5750523447490519431cd052c33 in term 1.
I20260809 07:56:48.378315 18476 leader_election.cc:290] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [CANDIDATE]: Term 1 election: Requested vote from peers f0f3a3af3f474a77a91a7149b5c75150 (127.26.37.62:36135), d3b24f399b0147b0bcb76cb4416063e8 (127.26.37.60:46289)
I20260809 07:56:48.378680 18266 leader_election.cc:304] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aee2b5750523447490519431cd052c33, d3b24f399b0147b0bcb76cb4416063e8; no voters:
I20260809 07:56:48.379025 18476 raft_consensus.cc:2804] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:56:48.379253 18476 raft_consensus.cc:697] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 1 LEADER]: Becoming Leader. State: Replica: aee2b5750523447490519431cd052c33, State: Running, Role: LEADER
I20260809 07:56:48.379953 18476 consensus_queue.cc:237] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.384863 18477 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee2b5750523447490519431cd052c33. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.385128 18477 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:48.387399 18229 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } attempt: 5
I20260809 07:56:48.396631 18487 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
Aug 09 07:56:48 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262206, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:48.413594 18487 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:56:48.425599 18229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: Bootstrap starting.
I20260809 07:56:48.429664 18229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:48.430987 18229 log.cc:826] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:48.441520 18229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: No bootstrap required, opened a new log
I20260809 07:56:48.445416 18401 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 1 FOLLOWER]: Refusing update from remote peer aee2b5750523447490519431cd052c33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:56:48.446470 18477 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:56:48.450028 18229 raft_consensus.cc:359] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.450346 18229 raft_consensus.cc:385] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:48.450469 18229 raft_consensus.cc:740] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f3a3af3f474a77a91a7149b5c75150, State: Initialized, Role: FOLLOWER
I20260809 07:56:48.451254 18229 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } }
I20260809 07:56:48.446164 18243 master_runner.cc:187] Error getting master registration for 127.26.37.62:36135: OK, instance_id { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" instance_seqno: 1786262206975940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:56:48.456111 18243 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:56:48.465955 18229 sys_catalog.cc:565] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:48.466022 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aee2b5750523447490519431cd052c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0f3a3af3f474a77a91a7149b5c75150" is_pre_election: true
W20260809 07:56:48.466315 18378 tablet.cc:2406] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:56:48.466327 18197 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:56:48.467629 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aee2b5750523447490519431cd052c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0f3a3af3f474a77a91a7149b5c75150"
I20260809 07:56:48.471050 18197 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 1 FOLLOWER]: Refusing update from remote peer aee2b5750523447490519431cd052c33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:56:48.465497 18503 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.471869 18503 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:48.472157 18477 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:56:48.473908 18454 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee2b5750523447490519431cd052c33. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.474192 18454 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:48.476030 18498 mvcc.cc:204] Tried to move back new op lower bound from 7316530005740724224 to 7316530005532569600. Current Snapshot: MvccSnapshot[applied={T|T < 7316530005740724224}]
I20260809 07:56:48.481639 18477 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.481838 18477 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:48.484133 18502 mvcc.cc:204] Tried to move back new op lower bound from 7316530005740724224 to 7316530005532569600. Current Snapshot: MvccSnapshot[applied={T|T < 7316530005740724224}]
I20260809 07:56:48.484745 18454 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.484932 18454 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:48.489650 18487 catalog_manager.cc:1383] Generated new cluster ID: df1b6604389d4a93878274606be697e6
I20260809 07:56:48.489775 18487 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:56:48.520161 18476 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.520498 18476 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:48.521430 18503 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee2b5750523447490519431cd052c33. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.521816 18503 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:48.523061 18503 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } }
I20260809 07:56:48.523306 18503 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:48.535683 18487 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:56:48.537930 18487 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:56:48.541122 18522 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:56:48.568658 18487 catalog_manager.cc:6092] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33: Generated new TSK 0
I20260809 07:56:48.570916 18487 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:48.764616 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:49.140200 18530 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:49.140514 18530 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:49.140645 18530 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:49.149209 18530 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:49.149471 18530 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:49.160563 18530 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:49.162464 18530 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:49.163836 18530 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:49.173699 18537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:49.173755 18535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:49.174185 18540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:49.175276 18530 server_base.cc:1061] running on GCE node
I20260809 07:56:49.175806 18530 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:49.175997 18530 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:49.176090 18530 hybrid_clock.cc:648] HybridClock initialized: now 1786262209176053 us; error 0 us; skew 500 ppm
I20260809 07:56:49.176371 18530 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:49.179579 18530 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:49.181164 18530 webserver.cc:533] Webserver started at http://127.26.37.1:41485/ using document root <none> and password file <none>
I20260809 07:56:49.181912 18530 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:49.182060 18530 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:49.182401 18530 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:49.184715 18530 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "abcfa8f41f1b4c2cb540e01bc0f061ec"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:49.185601 18530 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "abcfa8f41f1b4c2cb540e01bc0f061ec"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:49.190862 18530 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 07:56:49.194178 18547 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:49.195464 18530 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:56:49.195647 18530 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "abcfa8f41f1b4c2cb540e01bc0f061ec"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:49.195824 18530 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:49.215185 18530 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:49.218962 18530 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:49.219254 18530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:49.220144 18530 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:49.221892 18530 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:49.221994 18530 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:49.222143 18530 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:49.222224 18530 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:49.260982 18530 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:37281
I20260809 07:56:49.261015 18705 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:37281 every 8 connection(s)
I20260809 07:56:49.262534 18530 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:49.272069 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18530
I20260809 07:56:49.272308 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:49.298043 18708 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36135
I20260809 07:56:49.298291 18707 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46289
I20260809 07:56:49.298530 18707 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:49.298537 18708 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:49.299434 18709 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38325
I20260809 07:56:49.299630 18709 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:49.299945 18708 heartbeater.cc:507] Master 127.26.37.62:36135 requested a full tablet report, sending...
I20260809 07:56:49.300227 18709 heartbeater.cc:507] Master 127.26.37.61:38325 requested a full tablet report, sending...
I20260809 07:56:49.300483 18707 heartbeater.cc:507] Master 127.26.37.60:46289 requested a full tablet report, sending...
I20260809 07:56:49.302532 18288 ts_manager.cc:194] Registered new tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:49.302774 18186 ts_manager.cc:194] Registered new tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:49.302750 18391 ts_manager.cc:194] Registered new tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:49.304454 18288 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:50397
I20260809 07:56:49.317203 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:49.339773 18471 catalog_manager.cc:1295] Loaded cluster ID: df1b6604389d4a93878274606be697e6
I20260809 07:56:49.339895 18471 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: loading cluster ID for follower catalog manager: success
I20260809 07:56:49.343981 18471 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: acquiring CA information for follower catalog manager: success
I20260809 07:56:49.346637 18471 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for kudu/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:56:49.470630 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:40663
--webserver_interface=127.26.37.59
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:49.546093 18522 catalog_manager.cc:1295] Loaded cluster ID: df1b6604389d4a93878274606be697e6
I20260809 07:56:49.546247 18522 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: loading cluster ID for follower catalog manager: success
I20260809 07:56:49.550184 18522 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: acquiring CA information for follower catalog manager: success
I20260809 07:56:49.552628 18522 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:56:49.835557 18724 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:49.835897 18724 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:49.836088 18724 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:49.844708 18724 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:49.844805 18724 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:49.844914 18724 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:49.845016 18724 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:49.855683 18724 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:49.857452 18724 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:49.858911 18724 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:49.867712 18733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:49.867949 18734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:49.869659 18737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:49.871053 18724 server_base.cc:1061] running on GCE node
I20260809 07:56:49.871868 18724 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:49.872085 18724 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:49.872188 18724 hybrid_clock.cc:648] HybridClock initialized: now 1786262209872163 us; error 0 us; skew 500 ppm
I20260809 07:56:49.872641 18724 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:49.876479 18724 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:56:49.878685 18724 webserver.cc:533] Webserver started at http://127.26.37.59:33017/ using document root <none> and password file <none>
I20260809 07:56:49.879483 18724 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:49.879622 18724 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:49.880017 18724 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:56:49.882778 18724 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "bef41566c9a1479eb9be44a5df49282d"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:49.883651 18724 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "bef41566c9a1479eb9be44a5df49282d"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:49.889770 18724 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260809 07:56:49.893294 18745 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:49.894747 18724 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:56:49.894963 18724 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "bef41566c9a1479eb9be44a5df49282d"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:49.895161 18724 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:49.909994 18724 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:49.914099 18724 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:49.914415 18724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:49.935235 18724 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:40663
I20260809 07:56:49.935269 18824 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:40663 every 8 connection(s)
I20260809 07:56:49.936849 18724 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:56:49.941082 18826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:56:49.945813 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 18724
I20260809 07:56:49.947870 18826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:49.968532 18826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:49.981702 18826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:56:49.995397 18826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:56:49 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262209, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:50.011770 18826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Bootstrap starting.
I20260809 07:56:50.014869 18826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Neither blocks nor log segments found. Creating new log.
I20260809 07:56:50.015882 18826 log.cc:826] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:50.018411 18826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: No bootstrap required, opened a new log
I20260809 07:56:50.023806 18826 raft_consensus.cc:359] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } }
I20260809 07:56:50.024099 18826 raft_consensus.cc:385] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:56:50.024199 18826 raft_consensus.cc:740] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef41566c9a1479eb9be44a5df49282d, State: Initialized, Role: FOLLOWER
I20260809 07:56:50.024709 18826 consensus_queue.cc:260] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } }
I20260809 07:56:50.025558 18840 sys_catalog.cc:455] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } } }
I20260809 07:56:50.025936 18840 sys_catalog.cc:458] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:50.026631 18826 sys_catalog.cc:565] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:56:50.035606 18858 catalog_manager.cc:1594] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:56:50.035738 18858 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:56:50.042022 18724 master_runner.cc:429] Detected that this master bef41566c9a1479eb9be44a5df49282d is joining an existing cluster
I20260809 07:56:50.042173 18724 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:56:50.043201 18724 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:56:50.043565 18724 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:56:50.043748 18724 tablet_replica.cc:333] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: stopping tablet replica
I20260809 07:56:50.045275 18724 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:40663
Aug 09 07:56:50 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262207, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:56:50.097227 18288 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:40663: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bef41566c9a1479eb9be44a5df49282d"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 40663
}
attrs {
promote: true
}
}
dest_uuid: "aee2b5750523447490519431cd052c33"
cas_config_opid_index: -1
I20260809 07:56:50.098546 18288 consensus_queue.cc:237] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } }
I20260809 07:56:50.101660 18198 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 1 FOLLOWER]: Refusing update from remote peer aee2b5750523447490519431cd052c33: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260809 07:56:50.102190 18268 consensus_peers.cc:597] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 -> Peer bef41566c9a1479eb9be44a5df49282d (127.26.37.59:40663): Couldn't send request to peer bef41566c9a1479eb9be44a5df49282d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:56:50.102813 18477 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:56:50.103533 18401 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 1 FOLLOWER]: Refusing update from remote peer aee2b5750523447490519431cd052c33: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:56:50.111927 18876 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:56:50.115512 18876 raft_consensus.cc:2955] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bef41566c9a1479eb9be44a5df49282d (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.116564 18401 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bef41566c9a1479eb9be44a5df49282d (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.119917 18876 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:40663
I20260809 07:56:50.120555 18724 master.cc:562] Master@127.26.37.59:40663 shutting down...
I20260809 07:56:50.123615 18876 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.123929 18876 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
W20260809 07:56:50.155380 17700 connection.cc:573] client connection to 127.26.37.59:40663 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:56:50.159422 18477 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.159727 18477 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:50.159936 18724 master.cc:584] Master@127.26.37.59:40663 shutdown complete.
I20260809 07:56:50.160027 18724 master_runner.cc:306] Clearing existing system tablet
I20260809 07:56:50.160490 18724 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:50.160616 18724 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:50.160780 18198 raft_consensus.cc:2955] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bef41566c9a1479eb9be44a5df49282d (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.167707 18885 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.168910 18724 fs_manager.cc:714] Time spent opening directory manager: real 0.007s user 0.002s sys 0.001s
I20260809 07:56:50.168922 18885 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:50.173465 18892 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.173938 18892 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:50.174826 18893 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:50.174634 18879 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.174968 18879 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:50.175333 18724 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:50.175613 18724 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.004s sys 0.001s
I20260809 07:56:50.175725 18724 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "bef41566c9a1479eb9be44a5df49282d"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:50.178939 18724 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:56:50.183456 18724 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:56:50.183638 18724 log.cc:1199] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:56:50.184082 18724 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Deleting consensus metadata
I20260809 07:56:50.184350 18724 master_runner.cc:316] Copying system tablet from 127.26.37.61:38325
I20260809 07:56:50.186766 18724 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.61:38325
I20260809 07:56:50.199285 18313 tablet_copy_service.cc:140] P aee2b5750523447490519431cd052c33: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bef41566c9a1479eb9be44a5df49282d ({username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:38014)
I20260809 07:56:50.199628 18313 tablet_copy_service.cc:161] P aee2b5750523447490519431cd052c33: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bef41566c9a1479eb9be44a5df49282d at {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:38014: session id = bef41566c9a1479eb9be44a5df49282d-00000000000000000000000000000000
I20260809 07:56:50.204041 18313 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:56:50.206583 18724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:56:50.211261 18724 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: tablet copy: Starting download of 0 data blocks...
I20260809 07:56:50.211555 18724 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: tablet copy: Starting download of 1 WAL segments...
I20260809 07:56:50.214394 18724 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:56:50.217538 18313 tablet_copy_service.cc:342] P aee2b5750523447490519431cd052c33: Request end of tablet copy session bef41566c9a1479eb9be44a5df49282d-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:38014
I20260809 07:56:50.217734 18313 tablet_copy_service.cc:434] P aee2b5750523447490519431cd052c33: ending tablet copy session bef41566c9a1479eb9be44a5df49282d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bef41566c9a1479eb9be44a5df49282d
I20260809 07:56:50.228825 18724 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:50.231726 18907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:50.232931 18908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:50.233567 18910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:50.233819 18724 server_base.cc:1061] running on GCE node
I20260809 07:56:50.234158 18724 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:50.234252 18724 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:50.234320 18724 hybrid_clock.cc:648] HybridClock initialized: now 1786262210234282 us; error 0 us; skew 500 ppm
I20260809 07:56:50.234505 18724 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:50 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262210, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:50.236593 18724 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:56:50.246417 18724 webserver.cc:533] Webserver started at http://127.26.37.59:45235/ using document root <none> and password file <none>
I20260809 07:56:50.246835 18724 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:50.246968 18724 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:50.249512 18724 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:56:50.251451 18917 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:50.251945 18724 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 07:56:50.252104 18724 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "bef41566c9a1479eb9be44a5df49282d"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:50.252272 18724 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:50.266955 18724 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:50.270845 18724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:50.295572 18724 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:40663
I20260809 07:56:50.295634 18993 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:40663 every 8 connection(s)
I20260809 07:56:50.296583 18724 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:56:50.300359 18994 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:56:50.302986 18994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Bootstrap starting.
I20260809 07:56:50.306964 18709 heartbeater.cc:499] Master 127.26.37.61:38325 was elected leader, sending a full tablet report...
I20260809 07:56:50.321506 18994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:56:50.322199 18994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: Bootstrap complete.
I20260809 07:56:50.323253 18994 raft_consensus.cc:359] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } }
I20260809 07:56:50.323586 18994 raft_consensus.cc:740] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bef41566c9a1479eb9be44a5df49282d, State: Initialized, Role: LEARNER
I20260809 07:56:50.323944 18994 consensus_queue.cc:260] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } }
I20260809 07:56:50.324991 19001 sys_catalog.cc:455] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.325378 18994 sys_catalog.cc:565] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:50.325441 19001 sys_catalog.cc:458] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: This master's current role is: LEARNER
I20260809 07:56:50.336313 19013 catalog_manager.cc:1295] Loaded cluster ID: df1b6604389d4a93878274606be697e6
I20260809 07:56:50.336457 19013 catalog_manager.cc:1588] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: loading cluster ID for follower catalog manager: success
I20260809 07:56:50.339350 19013 catalog_manager.cc:1610] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: acquiring CA information for follower catalog manager: success
I20260809 07:56:50.341503 19013 catalog_manager.cc:1638] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:56:50.380970 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18530
Aug 09 07:56:50 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:56:50.399662 18701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:50.487890 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18530
I20260809 07:56:50.534648 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:37281
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--webserver_port=41485
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:50.588047 18956 raft_consensus.cc:1217] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:56:50.588773 19001 sys_catalog.cc:455] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee2b5750523447490519431cd052c33. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.589026 19001 sys_catalog.cc:458] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: This master's current role is: LEARNER
W20260809 07:56:50.906782 19018 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:50.907111 19018 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:50.907207 19018 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:50.915410 19018 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:50.915673 19018 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:50.926397 19018 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:37281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41485
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:50.928160 19018 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:50.929593 19018 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:50.939788 19024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:50.939888 19025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:50.940243 19031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:50.941295 19018 server_base.cc:1061] running on GCE node
I20260809 07:56:50.941901 19018 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:50.942080 19018 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:50.942186 19018 hybrid_clock.cc:648] HybridClock initialized: now 1786262210942146 us; error 0 us; skew 500 ppm
I20260809 07:56:50.942521 19018 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:50 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262210, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:50.946072 19018 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:50.947770 19018 webserver.cc:533] Webserver started at http://127.26.37.1:41485/ using document root <none> and password file <none>
I20260809 07:56:50.948557 19018 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:50.948726 19018 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:50.954651 19018 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:56:50.957768 19042 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:50.959116 19018 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:56:50.959303 19018 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "abcfa8f41f1b4c2cb540e01bc0f061ec"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:50.960052 19018 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:50.980136 19018 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:50.984861 19018 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:50.985181 19018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:50.985666 18879 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: true } } }
I20260809 07:56:50.985960 18879 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:50.992621 19018 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:50.997251 19018 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:50.997287 19047 raft_consensus.cc:1064] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33: attempting to promote NON_VOTER bef41566c9a1479eb9be44a5df49282d to VOTER
I20260809 07:56:50.997383 19018 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:50.997501 19018 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:50.997576 19018 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:50.998406 19047 consensus_queue.cc:237] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } }
I20260809 07:56:51.001673 18198 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 1 FOLLOWER]: Refusing update from remote peer aee2b5750523447490519431cd052c33: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:51.001667 18401 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 1 FOLLOWER]: Refusing update from remote peer aee2b5750523447490519431cd052c33: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:51.001698 18956 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 1 LEARNER]: Refusing update from remote peer aee2b5750523447490519431cd052c33: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:56:51.002640 18879 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:51.003024 19047 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:51.004889 19048 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:56:51.006014 19047 raft_consensus.cc:2955] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bef41566c9a1479eb9be44a5df49282d (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.006723 18401 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bef41566c9a1479eb9be44a5df49282d (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.010004 18879 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.010244 18879 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:51.010308 19047 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.010506 19047 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:51.011153 18198 raft_consensus.cc:2955] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bef41566c9a1479eb9be44a5df49282d (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.010509 18879 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.016865 18879 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:51.018893 19048 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.019235 19048 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:51.020169 18956 raft_consensus.cc:2955] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bef41566c9a1479eb9be44a5df49282d (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.026573 19001 sys_catalog.cc:455] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.026844 19001 sys_catalog.cc:458] T 00000000000000000000000000000000 P bef41566c9a1479eb9be44a5df49282d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:51.027630 19058 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.027968 19058 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b24f399b0147b0bcb76cb4416063e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:51.028638 19059 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.028946 19059 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:51.061141 19018 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:37281
I20260809 07:56:51.061239 19198 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:37281 every 8 connection(s)
I20260809 07:56:51.062779 19018 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:51.064046 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19018
Aug 09 07:56:51 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262210, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:51 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262210, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:56:51 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262210, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:51 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262210, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:51.123994 19201 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36135
I20260809 07:56:51.124084 19199 heartbeater.cc:344] Connected to a master server at 127.26.37.59:40663
I20260809 07:56:51.124401 19200 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46289
I20260809 07:56:51.124533 19199 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:51.124580 19200 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:51.124531 19201 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:51.125891 19200 heartbeater.cc:507] Master 127.26.37.60:46289 requested a full tablet report, sending...
I20260809 07:56:51.125921 19199 heartbeater.cc:507] Master 127.26.37.59:40663 requested a full tablet report, sending...
I20260809 07:56:51.126163 19201 heartbeater.cc:507] Master 127.26.37.62:36135 requested a full tablet report, sending...
I20260809 07:56:51.126462 19202 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38325
I20260809 07:56:51.126608 19202 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:51.127383 19202 heartbeater.cc:507] Master 127.26.37.61:38325 requested a full tablet report, sending...
I20260809 07:56:51.128587 18186 ts_manager.cc:194] Re-registered known tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:51.128583 18391 ts_manager.cc:194] Re-registered known tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:51.128742 18288 ts_manager.cc:194] Re-registered known tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:51.129215 18939 ts_manager.cc:194] Registered new tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:51.129750 18288 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:50843
I20260809 07:56:51.132829 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:51.140348 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:56:51.140529 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18141
I20260809 07:56:51.141549 18224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:51.257119 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18141
I20260809 07:56:51.285179 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36135
--webserver_interface=127.26.37.62
--webserver_port=34781
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:51.627902 18269 consensus_peers.cc:597] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 -> Peer f0f3a3af3f474a77a91a7149b5c75150 (127.26.37.62:36135): Couldn't send request to peer f0f3a3af3f474a77a91a7149b5c75150. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:36135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:56:51.653833 19221 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:51.654203 19221 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:51.654372 19221 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:51.662600 19221 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:56:51.662743 19221 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:51.662866 19221 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:56:51.662957 19221 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:56:51.673604 19221 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=34781
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:51.675215 19221 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:51.676630 19221 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:51.684849 19233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:51.684849 19232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:51.685452 19235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:51.686861 19221 server_base.cc:1061] running on GCE node
I20260809 07:56:51.687659 19221 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:51.687836 19221 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:51.687938 19221 hybrid_clock.cc:648] HybridClock initialized: now 1786262211687898 us; error 0 us; skew 500 ppm
I20260809 07:56:51.688300 19221 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:51 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262211, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:51.691879 19221 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:56:51.693725 19221 webserver.cc:533] Webserver started at http://127.26.37.62:34781/ using document root <none> and password file <none>
I20260809 07:56:51.694391 19221 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:51.694501 19221 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:51.699580 19221 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 07:56:51.702345 19242 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:51.703636 19221 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:51.703817 19221 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "f0f3a3af3f474a77a91a7149b5c75150"
format_stamp: "Formatted at 2026-08-09 07:56:46 on dist-test-slave-bw88"
I20260809 07:56:51.704524 19221 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:51.730290 19221 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:51.734515 19221 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:51.734822 19221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:51.756130 19221 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36135
I20260809 07:56:51.756173 19320 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36135 every 8 connection(s)
I20260809 07:56:51.757936 19221 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:56:51.765758 19321 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:56:51.767345 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19221
I20260809 07:56:51.767664 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19018
I20260809 07:56:51.769582 19321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: Bootstrap starting.
I20260809 07:56:51.784253 19193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:51.788611 19321 log.cc:826] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: Log is configured to *not* fsync() on all Append() calls
I20260809 07:56:51.801641 19321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:56:51.802274 19321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: Bootstrap complete.
I20260809 07:56:51.812345 19321 raft_consensus.cc:359] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } }
I20260809 07:56:51.813280 19321 raft_consensus.cc:740] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f3a3af3f474a77a91a7149b5c75150, State: Initialized, Role: FOLLOWER
I20260809 07:56:51.814119 19321 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } }
I20260809 07:56:51.814952 19331 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:51.815234 19331 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:56:51.816193 19321 sys_catalog.cc:565] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:56:51.828958 19349 catalog_manager.cc:1295] Loaded cluster ID: df1b6604389d4a93878274606be697e6
I20260809 07:56:51.829169 19349 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: loading cluster ID for follower catalog manager: success
I20260809 07:56:51.834609 19349 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: acquiring CA information for follower catalog manager: success
I20260809 07:56:51.838006 19349 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:56:51 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262211, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:56:51.918167 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19018
I20260809 07:56:51.953011 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:37281
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--webserver_port=41485
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:56:52.135994 19047 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:52.136842 19047 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:52.157119 19331 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee2b5750523447490519431cd052c33. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:52.157423 19331 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f3a3af3f474a77a91a7149b5c75150 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:56:52.339984 19363 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:52.340296 19363 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:52.340493 19363 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:52.349220 19363 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:52.349474 19363 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:52.359547 19363 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:37281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41485
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:52.361047 19363 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:52.362439 19363 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:52.371769 19374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:52.372372 19376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:52.372956 19380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:52.373716 19363 server_base.cc:1061] running on GCE node
I20260809 07:56:52.374254 19363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:52.374493 19363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:52.374606 19363 hybrid_clock.cc:648] HybridClock initialized: now 1786262212374560 us; error 0 us; skew 500 ppm
I20260809 07:56:52.374910 19363 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:52 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262212, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:52.378224 19363 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:52.379962 19363 webserver.cc:533] Webserver started at http://127.26.37.1:41485/ using document root <none> and password file <none>
I20260809 07:56:52.380736 19363 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:52.380882 19363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:52.386292 19363 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:56:52.388883 19390 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:52.390304 19363 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:52.390489 19363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "abcfa8f41f1b4c2cb540e01bc0f061ec"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:52.391260 19363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:52.418946 19363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:52.427055 19363 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:52.427289 19363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:52.428179 19363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:52.429786 19363 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:52.429877 19363 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:52.429955 19363 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:52.430007 19363 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:52.464766 19363 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:37281
I20260809 07:56:52.464860 19545 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:37281 every 8 connection(s)
I20260809 07:56:52.466481 19363 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:52.471684 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19363
Aug 09 07:56:52 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:56:52 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262212, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:56:52 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262212, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:56:52 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262212, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:56:52 dist-test-slave-bw88 krb5kdc[17702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262212, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:56:52.517401 19548 heartbeater.cc:344] Connected to a master server at 127.26.37.60:46289
I20260809 07:56:52.517802 19547 heartbeater.cc:344] Connected to a master server at 127.26.37.59:40663
I20260809 07:56:52.517916 19548 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:52.517937 19547 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:52.519006 19549 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36135
I20260809 07:56:52.519089 19547 heartbeater.cc:507] Master 127.26.37.59:40663 requested a full tablet report, sending...
I20260809 07:56:52.519213 19549 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:52.519333 19548 heartbeater.cc:507] Master 127.26.37.60:46289 requested a full tablet report, sending...
I20260809 07:56:52.519752 19549 heartbeater.cc:507] Master 127.26.37.62:36135 requested a full tablet report, sending...
I20260809 07:56:52.521294 18391 ts_manager.cc:194] Re-registered known tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:52.522022 19271 ts_manager.cc:194] Registered new tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:52.523917 18939 ts_manager.cc:194] Re-registered known tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:52.525467 19550 heartbeater.cc:344] Connected to a master server at 127.26.37.61:38325
I20260809 07:56:52.525619 19550 heartbeater.cc:461] Registering TS with master...
I20260809 07:56:52.526192 19550 heartbeater.cc:507] Master 127.26.37.61:38325 requested a full tablet report, sending...
I20260809 07:56:52.527159 18288 ts_manager.cc:194] Re-registered known tserver with Master: abcfa8f41f1b4c2cb540e01bc0f061ec (127.26.37.1:37281)
I20260809 07:56:52.528208 18288 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:56427
I20260809 07:56:52.628722 19364 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aee2b5750523447490519431cd052c33" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f0f3a3af3f474a77a91a7149b5c75150" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36135 } } peers { permanent_uuid: "aee2b5750523447490519431cd052c33" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 38325 } } peers { permanent_uuid: "d3b24f399b0147b0bcb76cb4416063e8" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 46289 } } peers { permanent_uuid: "bef41566c9a1479eb9be44a5df49282d" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 40663 } attrs { promote: false } } }
I20260809 07:56:52.629086 19364 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee2b5750523447490519431cd052c33 [sys.catalog]: This master's current role is: LEADER
I20260809 07:56:52.649379 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19363
W20260809 07:56:52.650815 19572 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:52.647188 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.26.37.1:37281)
0809 07:56:52.647871 (+ 683us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:52.647878 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:52.647910 (+ 32us) server_negotiation.cc:408] Connection header received
0809 07:56:52.647993 (+ 83us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:52.647999 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:52.648249 (+ 250us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:52.648413 (+ 164us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:52.649261 (+ 848us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44094: BlockingRecv error: recv got EOF from 127.0.0.1:44094 (error 108)
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":367,"threads_started":1}
W20260809 07:56:52.651671 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19363: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:37281: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:52.651803 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19363
I20260809 07:56:52.677567 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:37281
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--webserver_port=41485
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:53.036671 19574 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:53.036972 19574 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:53.037077 19574 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:56:53.037137 19574 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:53.045441 19574 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:53.045651 19574 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:53.055696 19574 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:37281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41485
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:53.057253 19574 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:53.058645 19574 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:53.068325 19580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:53.068305 19579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:53.069315 19585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:53.069777 19574 server_base.cc:1061] running on GCE node
I20260809 07:56:53.070279 19574 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:53.070405 19574 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:53.070477 19574 hybrid_clock.cc:648] HybridClock initialized: now 1786262213070448 us; error 0 us; skew 500 ppm
I20260809 07:56:53.070698 19574 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:53 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262213, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:53.074190 19574 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:53.075766 19574 webserver.cc:533] Webserver started at http://127.26.37.1:41485/ using document root <none> and password file <none>
I20260809 07:56:53.076418 19574 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:53.076500 19574 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:53.082166 19574 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260809 07:56:53.084928 19591 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:53.086671 19574 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:53.086848 19574 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "abcfa8f41f1b4c2cb540e01bc0f061ec"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:53.087620 19574 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:53.111819 19574 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:53.115772 19574 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:53.116081 19574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:53.117054 19574 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:53.118729 19574 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:53.118806 19574 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:53.118956 19574 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:53.119017 19574 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:56:53.119489 19574 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:56:53.119580 19574 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:56:53.119683 19574 fault_injection.cc:59] injecting fault for kudu (pid 19574): FLAGS_fault_before_start (process will exit)
I20260809 07:56:53.144222 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:37281
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289
--webserver_port=41485
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:56:53.507692 19606 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:56:53.508056 19606 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:56:53.508221 19606 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:56:53.516443 19606 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:56:53.516690 19606 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:56:53.527223 19606 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:37281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=41485
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:36135,127.26.37.61:38325,127.26.37.60:46289,127.26.37.59:40663
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:56:53.528889 19606 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:56:53.530359 19606 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:56:53.540374 19615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:53.540320 19616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:56:53.540601 19618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:56:53.542012 19606 server_base.cc:1061] running on GCE node
I20260809 07:56:53.542542 19606 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:56:53.542685 19606 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:56:53.542747 19606 hybrid_clock.cc:648] HybridClock initialized: now 1786262213542729 us; error 0 us; skew 500 ppm
I20260809 07:56:53.542989 19606 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:56:53 dist-test-slave-bw88 krb5kdc[17702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262213, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:56:53.546406 19606 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:56:53.548010 19606 webserver.cc:533] Webserver started at http://127.26.37.1:41485/ using document root <none> and password file <none>
I20260809 07:56:53.548697 19606 fs_manager.cc:362] Metadata directory not provided
I20260809 07:56:53.548794 19606 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:56:53.554477 19606 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:56:53.557328 19630 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:53.558786 19606 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:56:53.558964 19606 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "abcfa8f41f1b4c2cb540e01bc0f061ec"
format_stamp: "Formatted at 2026-08-09 07:56:49 on dist-test-slave-bw88"
I20260809 07:56:53.559759 19606 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:56:53.577226 19606 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:56:53.581311 19606 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:56:53.581669 19606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:56:53.582684 19606 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:56:53.584359 19606 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:56:53.584507 19606 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:53.584645 19606 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:56:53.584743 19606 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:56:53.625309 19606 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:37281
I20260809 07:56:53.625404 19804 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:37281 every 8 connection(s)
I20260809 07:56:53.627183 19606 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:56:53.631755 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 19606
I20260809 07:56:53.632129 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19606
W20260809 07:56:53.649219 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19606: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:37281: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:56:53.649416 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19606
I20260809 07:56:53.675694 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19221
W20260809 07:56:53.676301 19609 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:53.648226 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:59771 (local address 127.26.37.62:36135)
0809 07:56:53.648646 (+ 420us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:53.648653 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:53.649469 (+ 816us) server_negotiation.cc:408] Connection header received
0809 07:56:53.675273 (+ 25804us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:59771: BlockingRecv error: recv got EOF from 127.26.37.1:59771 (error 108)
Metrics: {"server-negotiator.queue_time_us":266}
W20260809 07:56:53.676357 19816 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:53.631046 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:58501 (local address 127.26.37.60:46289)
0809 07:56:53.642726 (+ 11680us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:53.642735 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:53.642763 (+ 28us) server_negotiation.cc:408] Connection header received
0809 07:56:53.642854 (+ 91us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:53.642863 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:53.642965 (+ 102us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:53.643128 (+ 163us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:53.675328 (+ 32200us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:58501: BlockingRecv error: recv got EOF from 127.26.37.1:58501 (error 108)
Metrics: {"server-negotiator.queue_time_us":11521,"thread_start_us":12353,"threads_started":1}
W20260809 07:56:53.676663 19810 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:53.630717 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:47999 (local address 127.26.37.59:40663)
0809 07:56:53.631201 (+ 484us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:53.631207 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:53.647898 (+ 16691us) server_negotiation.cc:408] Connection header received
0809 07:56:53.648059 (+ 161us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:53.648066 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:53.648175 (+ 109us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:53.648296 (+ 121us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:53.675894 (+ 27598us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:47999: BlockingRecv error: recv got EOF from 127.26.37.1:47999 (error 108)
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":2002,"threads_started":1}
W20260809 07:56:53.677989 19813 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:53.630828 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:38753 (local address 127.26.37.61:38325)
0809 07:56:53.644603 (+ 13775us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:53.644607 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:53.677286 (+ 32679us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:38753: BlockingRecv error: recv got EOF from 127.26.37.1:38753 (error 108)
Metrics: {"server-negotiator.queue_time_us":13656,"thread_start_us":13548,"threads_started":1}
W20260809 07:56:53.678535 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 19221: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:36135: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:56:53.678664 19609 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:56:53.677143 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.26.37.62:36135)
0809 07:56:53.677331 (+ 188us) server_negotiation.cc:207] Beginning negotiation
0809 07:56:53.677367 (+ 36us) server_negotiation.cc:400] Waiting for connection header
0809 07:56:53.677388 (+ 21us) server_negotiation.cc:408] Connection header received
0809 07:56:53.677461 (+ 73us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:56:53.677467 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:56:53.677566 (+ 99us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:56:53.677687 (+ 121us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:56:53.678533 (+ 846us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55918: BlockingRecv error: recv got EOF from 127.0.0.1:55918 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 07:56:53.678696 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 19221
I20260809 07:56:53.707206 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18243
I20260809 07:56:53.708403 18328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:53.823204 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18243
I20260809 07:56:53.853601 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18352
I20260809 07:56:53.854836 18429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:53.963025 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18352
I20260809 07:56:53.994261 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 18724
I20260809 07:56:53.995460 18986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:56:54.109777 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 18724
Shutting down hive metastore.
[ OK ] ExternalMiniClusterTest.TestAddMaster/9 (26895 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/10
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:56:54 dist-test-slave-bw88 krb5kdc[19837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:56:54 dist-test-slave-bw88 krb5kdc[19837](info): set up 2 sockets
Aug 09 07:56:54 dist-test-slave-bw88 krb5kdc[19837](info): commencing operation
krb5kdc: starting...
W20260809 07:56:56.810240 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.264s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:56:57 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:56:57Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:56:57Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:57:04: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:57:06 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262226, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:57:12 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262232, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:57:12 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262232, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:13.468331 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41995
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:13.815089 20267 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:13.815389 20267 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:13.815483 20267 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:13.823239 20267 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:13.823324 20267 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:13.823377 20267 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:13.823423 20267 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:13.833216 20267 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:13.834729 20267 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:13.835997 20267 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:13.844230 20275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:13.844106 20276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:13.844966 20278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:13.845155 20267 server_base.cc:1061] running on GCE node
I20260809 07:57:13.845904 20267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:13.847191 20267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:13.848464 20267 hybrid_clock.cc:648] HybridClock initialized: now 1786262233848397 us; error 34 us; skew 500 ppm
I20260809 07:57:13.848807 20267 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:13 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262233, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:13.852252 20267 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:57:13.853988 20267 webserver.cc:533] Webserver started at http://127.26.37.62:39645/ using document root <none> and password file <none>
I20260809 07:57:13.854749 20267 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:13.854950 20267 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:13.855369 20267 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:13.858062 20267 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "d2d5a364f6f944dfaf4595b5c7d960d8"
format_stamp: "Formatted at 2026-08-09 07:57:13 on dist-test-slave-bw88"
I20260809 07:57:13.858953 20267 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "d2d5a364f6f944dfaf4595b5c7d960d8"
format_stamp: "Formatted at 2026-08-09 07:57:13 on dist-test-slave-bw88"
I20260809 07:57:13.864646 20267 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:57:13.868211 20286 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:13.869630 20267 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:57:13.869825 20267 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "d2d5a364f6f944dfaf4595b5c7d960d8"
format_stamp: "Formatted at 2026-08-09 07:57:13 on dist-test-slave-bw88"
I20260809 07:57:13.870030 20267 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:13.895411 20267 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:13.902423 20267 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:13.902709 20267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:13.922851 20267 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41995
I20260809 07:57:13.922901 20360 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41995 every 8 connection(s)
I20260809 07:57:13.924266 20267 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:57:13.928601 20361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:13.930505 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20267
I20260809 07:57:13.930698 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:57:13.934090 20361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:13 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262233, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:13.957010 20361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:57:13.958384 20288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34577: connect: Connection refused (error 111)
W20260809 07:57:13.960067 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34577: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34577: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:14.009511 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } attempt: 1
W20260809 07:57:14.011612 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34577: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34577: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:14.072870 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:34577
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:14.079941 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } attempt: 2
W20260809 07:57:14.081918 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34577: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34577: connect: Connection refused (error 111)
I20260809 07:57:14.173233 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } attempt: 3
W20260809 07:57:14.175508 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34577: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34577: connect: Connection refused (error 111)
I20260809 07:57:14.318833 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } attempt: 4
W20260809 07:57:14.321179 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:34577: Network error: Client connection negotiation failed: client connection to 127.26.37.61:34577: connect: Connection refused (error 111)
W20260809 07:57:14.424154 20372 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:14.424485 20372 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:14.424642 20372 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:14.432824 20372 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:14.432917 20372 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:14.433008 20372 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:14.433095 20372 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:14.443171 20372 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:14.444757 20372 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:14.446115 20372 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:14.454484 20379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:14.454819 20378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:14.455220 20381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:14.455828 20372 server_base.cc:1061] running on GCE node
I20260809 07:57:14.456635 20372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:14.458161 20372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:14.459394 20372 hybrid_clock.cc:648] HybridClock initialized: now 1786262234459352 us; error 39 us; skew 500 ppm
I20260809 07:57:14.459743 20372 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:14 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262234, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:14.462931 20372 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:57:14.464577 20372 webserver.cc:533] Webserver started at http://127.26.37.61:42821/ using document root <none> and password file <none>
I20260809 07:57:14.465312 20372 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:14.465472 20372 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:14.465792 20372 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:14.468236 20372 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "c859add327ef4282843e216ad200f66b"
format_stamp: "Formatted at 2026-08-09 07:57:14 on dist-test-slave-bw88"
I20260809 07:57:14.469065 20372 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "c859add327ef4282843e216ad200f66b"
format_stamp: "Formatted at 2026-08-09 07:57:14 on dist-test-slave-bw88"
I20260809 07:57:14.474517 20372 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:57:14.477952 20390 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:14.479281 20372 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:57:14.479413 20372 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "c859add327ef4282843e216ad200f66b"
format_stamp: "Formatted at 2026-08-09 07:57:14 on dist-test-slave-bw88"
I20260809 07:57:14.479578 20372 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:14.520108 20372 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:14.523732 20372 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:14.523968 20372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:14.545323 20372 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:34577
I20260809 07:57:14.545454 20460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:34577 every 8 connection(s)
I20260809 07:57:14.547442 20372 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:57:14.552052 20461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:14.555837 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20372
I20260809 07:57:14.556039 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:57:14.558298 20461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:14 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262234, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:14.579867 20461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:14 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262234, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
I20260809 07:57:14.598332 20461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:57:14.599659 20394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
W20260809 07:57:14.601411 20461 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
I20260809 07:57:14.620463 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } attempt: 5
Aug 09 07:57:14 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262233, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:14.635318 20361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:57:14.637527 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:14.650768 20461 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 1
W20260809 07:57:14.653647 20461 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:57:14.688756 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 1
W20260809 07:57:14.690941 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:14.712281 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:41421
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:14.721861 20461 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 2
W20260809 07:57:14.723829 20461 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
I20260809 07:57:14.759263 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 2
W20260809 07:57:14.761408 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
I20260809 07:57:14.815099 20461 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 3
W20260809 07:57:14.817277 20461 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
I20260809 07:57:14.867651 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 3
W20260809 07:57:14.870025 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
I20260809 07:57:14.960636 20461 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 4
W20260809 07:57:14.962961 20461 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
I20260809 07:57:15.047266 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 4
W20260809 07:57:15.049433 20361 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:41421: Network error: Client connection negotiation failed: client connection to 127.26.37.60:41421: connect: Connection refused (error 111)
W20260809 07:57:15.075682 20471 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:15.076028 20471 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:15.076197 20471 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:15.084919 20471 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:15.085069 20471 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:15.085183 20471 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:15.085266 20471 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:15.097522 20471 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:41421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:15.099233 20471 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:15.100696 20471 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:15.109118 20482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:15.109249 20484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:15.109750 20487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:15.110589 20471 server_base.cc:1061] running on GCE node
I20260809 07:57:15.111312 20471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:15.112645 20471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:15.113839 20471 hybrid_clock.cc:648] HybridClock initialized: now 1786262235113803 us; error 25 us; skew 500 ppm
I20260809 07:57:15.114169 20471 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262235, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:15.117424 20471 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:57:15.119041 20471 webserver.cc:533] Webserver started at http://127.26.37.60:41663/ using document root <none> and password file <none>
I20260809 07:57:15.120078 20471 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:15.120209 20471 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:15.120626 20471 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:15.123797 20471 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "56dc1770c07d45d3b3838210865c856c"
format_stamp: "Formatted at 2026-08-09 07:57:15 on dist-test-slave-bw88"
I20260809 07:57:15.124603 20471 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "56dc1770c07d45d3b3838210865c856c"
format_stamp: "Formatted at 2026-08-09 07:57:15 on dist-test-slave-bw88"
I20260809 07:57:15.130038 20471 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:57:15.133327 20495 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:15.134811 20471 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:57:15.134995 20471 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "56dc1770c07d45d3b3838210865c856c"
format_stamp: "Formatted at 2026-08-09 07:57:15 on dist-test-slave-bw88"
I20260809 07:57:15.135174 20471 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:15.152449 20471 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:15.156355 20471 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:15.156663 20471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:15.177537 20471 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:41421
I20260809 07:57:15.177557 20569 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:41421 every 8 connection(s)
I20260809 07:57:15.179103 20471 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:57:15.183357 20570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:15.185662 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20471
I20260809 07:57:15.185866 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:57:15.191509 20570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262235, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:15.213604 20570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262235, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:15.229483 20570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262235, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:57:15.249467 20570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: Bootstrap starting.
I20260809 07:57:15.252767 20570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:15.254112 20570 log.cc:826] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:15.256666 20570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: No bootstrap required, opened a new log
I20260809 07:57:15.262034 20570 raft_consensus.cc:359] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.262212 20461 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 5
I20260809 07:57:15.262303 20570 raft_consensus.cc:385] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:15.262457 20570 raft_consensus.cc:740] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dc1770c07d45d3b3838210865c856c, State: Initialized, Role: FOLLOWER
I20260809 07:57:15.263186 20570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.264238 20587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.264582 20587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:15.265406 20570 sys_catalog.cc:565] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262234, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
W20260809 07:57:15.280687 20603 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:57:15.280810 20603 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:57:15.283610 20461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: Bootstrap starting.
I20260809 07:57:15.279371 20471 master_runner.cc:187] Error getting master registration for 127.26.37.62:41995: OK, instance_id { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" instance_seqno: 1786262233917409 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:57:15.284606 20471 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:57:15.287989 20461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:15.289332 20461 log.cc:826] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:15.293084 20461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: No bootstrap required, opened a new log
I20260809 07:57:15.298610 20461 raft_consensus.cc:359] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.298868 20461 raft_consensus.cc:385] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:15.299062 20461 raft_consensus.cc:740] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c859add327ef4282843e216ad200f66b, State: Initialized, Role: FOLLOWER
I20260809 07:57:15.299692 20461 consensus_queue.cc:260] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.300776 20607 sys_catalog.cc:455] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.302170 20607 sys_catalog.cc:458] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:15.302852 20461 sys_catalog.cc:565] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:57:15.313505 20622 catalog_manager.cc:1594] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:57:15.313731 20622 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:57:15.310482 20372 master_runner.cc:187] Error getting master registration for 127.26.37.62:41995: OK, instance_id { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" instance_seqno: 1786262233917409 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:57:15.319217 20372 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:57:15.326713 20361 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } attempt: 5
Aug 09 07:57:15 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262233, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:57:15.350602 20361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: Bootstrap starting.
I20260809 07:57:15.355365 20361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:15.356920 20361 log.cc:826] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:15.360545 20361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: No bootstrap required, opened a new log
I20260809 07:57:15.369196 20361 raft_consensus.cc:359] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.369671 20361 raft_consensus.cc:385] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:15.369825 20361 raft_consensus.cc:740] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d5a364f6f944dfaf4595b5c7d960d8, State: Initialized, Role: FOLLOWER
I20260809 07:57:15.370743 20361 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.371903 20625 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.372278 20625 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:15.373373 20361 sys_catalog.cc:565] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:57:15.392989 20267 master_runner.cc:264] No leader master found from master d2d5a364f6f944dfaf4595b5c7d960d8
I20260809 07:57:15.393177 20267 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:57:15.399962 20638 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:57:15.400125 20638 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 07:57:15.406688 20294 tablet.cc:2406] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:57:15.408795 20501 tablet.cc:2406] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:57:15.526732 20399 tablet.cc:2406] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:57:15.545432 20587 raft_consensus.cc:493] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:57:15.545758 20587 raft_consensus.cc:515] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.547731 20587 leader_election.cc:290] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2d5a364f6f944dfaf4595b5c7d960d8 (127.26.37.62:41995), c859add327ef4282843e216ad200f66b (127.26.37.61:34577)
I20260809 07:57:15.548920 20429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56dc1770c07d45d3b3838210865c856c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c859add327ef4282843e216ad200f66b" is_pre_election: true
I20260809 07:57:15.549441 20429 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56dc1770c07d45d3b3838210865c856c in term 0.
I20260809 07:57:15.550385 20497 leader_election.cc:304] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56dc1770c07d45d3b3838210865c856c, c859add327ef4282843e216ad200f66b; no voters:
I20260809 07:57:15.551800 20326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56dc1770c07d45d3b3838210865c856c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" is_pre_election: true
I20260809 07:57:15.552999 20326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56dc1770c07d45d3b3838210865c856c in term 0.
I20260809 07:57:15.554049 20587 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:57:15.554200 20587 raft_consensus.cc:493] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:57:15.554302 20587 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:57:15.558120 20587 raft_consensus.cc:515] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.559337 20326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56dc1770c07d45d3b3838210865c856c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8"
I20260809 07:57:15.559548 20326 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:57:15.563199 20326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56dc1770c07d45d3b3838210865c856c in term 1.
I20260809 07:57:15.563794 20496 leader_election.cc:304] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56dc1770c07d45d3b3838210865c856c, d2d5a364f6f944dfaf4595b5c7d960d8; no voters:
I20260809 07:57:15.567504 20429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56dc1770c07d45d3b3838210865c856c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c859add327ef4282843e216ad200f66b"
I20260809 07:57:15.567720 20429 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:57:15.571391 20429 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56dc1770c07d45d3b3838210865c856c in term 1.
I20260809 07:57:15.572911 20587 leader_election.cc:290] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [CANDIDATE]: Term 1 election: Requested vote from peers d2d5a364f6f944dfaf4595b5c7d960d8 (127.26.37.62:41995), c859add327ef4282843e216ad200f66b (127.26.37.61:34577)
I20260809 07:57:15.573572 20587 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:57:15.573938 20587 raft_consensus.cc:697] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 1 LEADER]: Becoming Leader. State: Replica: 56dc1770c07d45d3b3838210865c856c, State: Running, Role: LEADER
I20260809 07:57:15.575362 20587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } }
I20260809 07:57:15.588829 20640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56dc1770c07d45d3b3838210865c856c. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.589386 20640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:15.596349 20642 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:57:15.599217 20642 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:57:15.619233 20429 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 1 FOLLOWER]: Refusing update from remote peer 56dc1770c07d45d3b3838210865c856c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:57:15.620384 20640 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:57:15.627919 20326 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 1 FOLLOWER]: Refusing update from remote peer 56dc1770c07d45d3b3838210865c856c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:57:15.628958 20640 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:57:15.652280 20607 sys_catalog.cc:455] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56dc1770c07d45d3b3838210865c856c. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.652591 20607 sys_catalog.cc:458] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:15.659118 20647 mvcc.cc:204] Tried to move back new op lower bound from 7316530117083750400 to 7316530116930236416. Current Snapshot: MvccSnapshot[applied={T|T < 7316530117083750400}]
I20260809 07:57:15.660749 20645 mvcc.cc:204] Tried to move back new op lower bound from 7316530117083750400 to 7316530116930236416. Current Snapshot: MvccSnapshot[applied={T|T < 7316530117083750400}]
I20260809 07:57:15.665804 20587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.666100 20587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:15.668164 20642 catalog_manager.cc:1383] Generated new cluster ID: a38988ebe5e1430ab1ffa4e6f8d2d2d5
I20260809 07:57:15.668277 20642 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:57:15.678206 20607 sys_catalog.cc:455] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.678465 20607 sys_catalog.cc:458] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:15.690563 20625 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56dc1770c07d45d3b3838210865c856c. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.690840 20625 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:15.702592 20625 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.702852 20625 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:15.703933 20640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } }
I20260809 07:57:15.704908 20640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:15.716470 20648 mvcc.cc:204] Tried to move back new op lower bound from 7316530117083750400 to 7316530116930236416. Current Snapshot: MvccSnapshot[applied={T|T < 7316530117083750400 or (T in {7316530117083750400})}]
I20260809 07:57:15.732612 20642 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:57:15.736171 20642 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:57:15.769322 20642 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: Generated new TSK 0
I20260809 07:57:15.770651 20642 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:15.990978 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:16.315721 20622 catalog_manager.cc:1295] Loaded cluster ID: a38988ebe5e1430ab1ffa4e6f8d2d2d5
I20260809 07:57:16.315820 20622 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: loading cluster ID for follower catalog manager: success
I20260809 07:57:16.318511 20622 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: acquiring CA information for follower catalog manager: success
I20260809 07:57:16.320248 20622 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:57:16.355988 20662 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:16.356357 20662 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:16.356540 20662 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:16.364599 20662 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:16.364872 20662 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:16.375157 20662 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:16.376885 20662 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:16.378413 20662 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:16.388433 20670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:16.388433 20669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:16.388532 20674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:16.389598 20662 server_base.cc:1061] running on GCE node
I20260809 07:57:16.390149 20662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:16.391060 20662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:16.392297 20662 hybrid_clock.cc:648] HybridClock initialized: now 1786262236392239 us; error 44 us; skew 500 ppm
I20260809 07:57:16.392637 20662 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:16 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262236, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:16.395953 20662 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:16.397585 20662 webserver.cc:533] Webserver started at http://127.26.37.1:45791/ using document root <none> and password file <none>
I20260809 07:57:16.398295 20662 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:16.398447 20662 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:16.398787 20662 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:16.401691 20662 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "c31f49d13b3e49f28f6113ad88b4f2f7"
format_stamp: "Formatted at 2026-08-09 07:57:16 on dist-test-slave-bw88"
I20260809 07:57:16.402534 20638 catalog_manager.cc:1295] Loaded cluster ID: a38988ebe5e1430ab1ffa4e6f8d2d2d5
I20260809 07:57:16.402626 20638 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: loading cluster ID for follower catalog manager: success
I20260809 07:57:16.402769 20662 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "c31f49d13b3e49f28f6113ad88b4f2f7"
format_stamp: "Formatted at 2026-08-09 07:57:16 on dist-test-slave-bw88"
I20260809 07:57:16.406499 20638 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: acquiring CA information for follower catalog manager: success
I20260809 07:57:16.408847 20638 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:57:16.409729 20662 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 07:57:16.412937 20685 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:16.414280 20662 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:16.414430 20662 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c31f49d13b3e49f28f6113ad88b4f2f7"
format_stamp: "Formatted at 2026-08-09 07:57:16 on dist-test-slave-bw88"
I20260809 07:57:16.414606 20662 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:16.440330 20662 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:16.443915 20662 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:16.444155 20662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:16.445025 20662 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:16.446686 20662 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:16.446763 20662 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:16.446868 20662 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:16.446914 20662 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:16.482527 20662 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35495
I20260809 07:57:16.482581 20857 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35495 every 8 connection(s)
I20260809 07:57:16.483970 20662 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:16.488023 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20662
I20260809 07:57:16.488255 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:57:16 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262236, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:57:16 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262236, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:16.514544 20859 heartbeater.cc:344] Connected to a master server at 127.26.37.60:41421
I20260809 07:57:16.515188 20859 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:16.516412 20859 heartbeater.cc:507] Master 127.26.37.60:41421 requested a full tablet report, sending...
Aug 09 07:57:16 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262236, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:16.519121 20520 ts_manager.cc:194] Registered new tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:16.520989 20520 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:57863
I20260809 07:57:16.523279 20860 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41995
I20260809 07:57:16.523439 20860 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:16.523805 20860 heartbeater.cc:507] Master 127.26.37.62:41995 requested a full tablet report, sending...
I20260809 07:57:16.525624 20309 ts_manager.cc:194] Registered new tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:16.525920 20861 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34577
I20260809 07:57:16.526053 20861 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:16.526391 20861 heartbeater.cc:507] Master 127.26.37.61:34577 requested a full tablet report, sending...
I20260809 07:57:16.527828 20415 ts_manager.cc:194] Registered new tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:16.530086 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:16.678781 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:38289
--webserver_interface=127.26.37.59
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:17.037285 20876 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:17.037680 20876 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:17.037827 20876 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:17.045959 20876 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:17.046047 20876 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:17.046173 20876 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:17.046241 20876 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:17.056779 20876 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:38289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:17.058539 20876 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:17.059930 20876 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:17.068105 20884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:17.068164 20886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:17.069060 20889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:17.070078 20876 server_base.cc:1061] running on GCE node
I20260809 07:57:17.070915 20876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:17.072222 20876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:17.073437 20876 hybrid_clock.cc:648] HybridClock initialized: now 1786262237073357 us; error 61 us; skew 500 ppm
I20260809 07:57:17.073748 20876 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262237, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:17.077188 20876 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:57:17.078897 20876 webserver.cc:533] Webserver started at http://127.26.37.59:45117/ using document root <none> and password file <none>
I20260809 07:57:17.079720 20876 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:17.079847 20876 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:17.080147 20876 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:17.082818 20876 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "2ef5d7cce34142e896a4cc37e6cc22ba"
format_stamp: "Formatted at 2026-08-09 07:57:17 on dist-test-slave-bw88"
I20260809 07:57:17.083562 20876 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "2ef5d7cce34142e896a4cc37e6cc22ba"
format_stamp: "Formatted at 2026-08-09 07:57:17 on dist-test-slave-bw88"
I20260809 07:57:17.091126 20876 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260809 07:57:17.095220 20898 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:17.096561 20876 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:57:17.096727 20876 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "2ef5d7cce34142e896a4cc37e6cc22ba"
format_stamp: "Formatted at 2026-08-09 07:57:17 on dist-test-slave-bw88"
I20260809 07:57:17.096911 20876 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:17.120308 20876 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:17.124037 20876 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:17.124305 20876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:17.145996 20876 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:38289
I20260809 07:57:17.146016 20971 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:38289 every 8 connection(s)
I20260809 07:57:17.147518 20876 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:57:17.151736 20973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:17.153955 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 20876
I20260809 07:57:17.158959 20973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262237, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:17.179163 20973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262237, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:17.192345 20973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262237, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:57:17.208346 20973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262237, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:57:17.224130 20973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Bootstrap starting.
I20260809 07:57:17.227241 20973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:17.228302 20973 log.cc:826] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:17.230808 20973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: No bootstrap required, opened a new log
I20260809 07:57:17.235860 20973 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } }
I20260809 07:57:17.236157 20973 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:17.236261 20973 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef5d7cce34142e896a4cc37e6cc22ba, State: Initialized, Role: FOLLOWER
I20260809 07:57:17.236905 20973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } }
I20260809 07:57:17.237831 20986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } } }
I20260809 07:57:17.238104 20986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:17.238772 20973 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:57:17.247876 21003 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:57:17.248014 21003 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:57:17.255040 20876 master_runner.cc:429] Detected that this master 2ef5d7cce34142e896a4cc37e6cc22ba is joining an existing cluster
I20260809 07:57:17.255177 20876 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:57:17.256064 20876 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:57:17.256393 20876 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:57:17.256575 20876 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: stopping tablet replica
I20260809 07:57:17.258323 20876 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:38289
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262235, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:57:17.312077 20520 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:38289: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 38289
}
attrs {
promote: true
}
}
dest_uuid: "56dc1770c07d45d3b3838210865c856c"
cas_config_opid_index: -1
I20260809 07:57:17.313822 20520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } }
I20260809 07:57:17.316987 20326 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 1 FOLLOWER]: Refusing update from remote peer 56dc1770c07d45d3b3838210865c856c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:57:17.317806 20653 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 07:57:17.318392 20499 consensus_peers.cc:597] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c -> Peer 2ef5d7cce34142e896a4cc37e6cc22ba (127.26.37.59:38289): Couldn't send request to peer 2ef5d7cce34142e896a4cc37e6cc22ba. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:57:17.319083 20428 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 1 FOLLOWER]: Refusing update from remote peer 56dc1770c07d45d3b3838210865c856c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:57:17.319669 20653 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:57:17.321585 21016 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2ef5d7cce34142e896a4cc37e6cc22ba (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.322402 20326 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2ef5d7cce34142e896a4cc37e6cc22ba (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.325776 21016 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:38289
I20260809 07:57:17.326571 20876 master.cc:562] Master@127.26.37.59:38289 shutting down...
I20260809 07:57:17.337918 20653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.338194 20653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:17.337998 21023 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.338369 21023 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:17.338922 20653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.339102 20653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:17.343398 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.343724 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:17.344170 20428 raft_consensus.cc:2955] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2ef5d7cce34142e896a4cc37e6cc22ba (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.347707 21030 sys_catalog.cc:455] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.348004 21030 sys_catalog.cc:458] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:57:17.364331 20899 connection.cc:466] server connection from 127.0.0.1:48310 torn down before Call kudu.master.MasterService.ListTables from 127.0.0.1:48310 (request call id 5) could send its response
I20260809 07:57:17.378346 20876 master.cc:584] Master@127.26.37.59:38289 shutdown complete.
I20260809 07:57:17.378449 20876 master_runner.cc:306] Clearing existing system tablet
I20260809 07:57:17.378934 20876 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:17.379043 20876 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:17.381930 20876 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 07:57:17.383584 21034 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:17.384068 20876 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:17.384357 20876 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:57:17.384444 20876 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "2ef5d7cce34142e896a4cc37e6cc22ba"
format_stamp: "Formatted at 2026-08-09 07:57:17 on dist-test-slave-bw88"
I20260809 07:57:17.389065 20876 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:57:17.396162 20876 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:57:17.396354 20876 log.cc:1199] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:57:17.396823 20876 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Deleting consensus metadata
I20260809 07:57:17.397118 20876 master_runner.cc:316] Copying system tablet from 127.26.37.60:41421
I20260809 07:57:17.399870 20876 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.60:41421
I20260809 07:57:17.413802 20543 tablet_copy_service.cc:140] P 56dc1770c07d45d3b3838210865c856c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2ef5d7cce34142e896a4cc37e6cc22ba ({username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:48502)
I20260809 07:57:17.414146 20543 tablet_copy_service.cc:161] P 56dc1770c07d45d3b3838210865c856c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2ef5d7cce34142e896a4cc37e6cc22ba at {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:48502: session id = 2ef5d7cce34142e896a4cc37e6cc22ba-00000000000000000000000000000000
I20260809 07:57:17.418516 20543 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:57:17.420854 20876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:57:17.425791 20876 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: tablet copy: Starting download of 0 data blocks...
I20260809 07:57:17.426069 20876 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: tablet copy: Starting download of 1 WAL segments...
I20260809 07:57:17.428661 20876 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:57:17.432153 20543 tablet_copy_service.cc:342] P 56dc1770c07d45d3b3838210865c856c: Request end of tablet copy session 2ef5d7cce34142e896a4cc37e6cc22ba-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:48502
I20260809 07:57:17.432384 20543 tablet_copy_service.cc:434] P 56dc1770c07d45d3b3838210865c856c: ending tablet copy session 2ef5d7cce34142e896a4cc37e6cc22ba-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2ef5d7cce34142e896a4cc37e6cc22ba
W20260809 07:57:17.439738 20876 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260809 07:57:17.443871 20876 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:17.446452 21046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:17.446832 21045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:17.448330 21049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:17.453967 20876 server_base.cc:1061] running on GCE node
I20260809 07:57:17.454270 20876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:17.454830 20876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:17.456041 20876 hybrid_clock.cc:648] HybridClock initialized: now 1786262237455991 us; error 35 us; skew 500 ppm
I20260809 07:57:17.456280 20876 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262237, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:17.458539 20876 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:57:17.459852 20876 webserver.cc:533] Webserver started at http://127.26.37.59:44869/ using document root <none> and password file <none>
I20260809 07:57:17.460182 20876 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:17.460285 20876 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:17.462664 20876 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 07:57:17.464329 21058 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:17.464756 20876 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 07:57:17.464893 20876 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "2ef5d7cce34142e896a4cc37e6cc22ba"
format_stamp: "Formatted at 2026-08-09 07:57:17 on dist-test-slave-bw88"
I20260809 07:57:17.465054 20876 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:17.490167 20876 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:17.493726 20876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:17.511368 20876 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:38289
I20260809 07:57:17.511400 21134 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:38289 every 8 connection(s)
I20260809 07:57:17.512567 20876 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:57:17.515892 21136 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:57:17.518435 21136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Bootstrap starting.
I20260809 07:57:17.524930 20859 heartbeater.cc:499] Master 127.26.37.60:41421 was elected leader, sending a full tablet report...
I20260809 07:57:17.540647 21136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:57:17.541129 21136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: Bootstrap complete.
I20260809 07:57:17.542184 21136 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } }
I20260809 07:57:17.542433 21136 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2ef5d7cce34142e896a4cc37e6cc22ba, State: Initialized, Role: LEARNER
I20260809 07:57:17.542673 21136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } }
I20260809 07:57:17.543565 21143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.543865 21136 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:57:17.543898 21143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: This master's current role is: LEARNER
I20260809 07:57:17.558614 21156 catalog_manager.cc:1295] Loaded cluster ID: a38988ebe5e1430ab1ffa4e6f8d2d2d5
I20260809 07:57:17.558724 21156 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: loading cluster ID for follower catalog manager: success
I20260809 07:57:17.561389 21156 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: acquiring CA information for follower catalog manager: success
I20260809 07:57:17.563335 21156 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:57:17.590158 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20662
Aug 09 07:57:17 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:57:17.608214 20847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:17.719780 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20662
I20260809 07:57:17.749845 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35495
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--webserver_port=45791
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:17.766777 21097 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:57:17.767666 21143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56dc1770c07d45d3b3838210865c856c. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:17.767923 21143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: This master's current role is: LEARNER
W20260809 07:57:18.120898 21163 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:18.121234 21163 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:18.121392 21163 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:18.130329 21163 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:18.130528 21163 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:18.142561 21163 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45791
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:18.144265 21163 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:18.145907 21163 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:18.156826 21168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:18.156863 21169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:18.157697 21172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:18.158321 21163 server_base.cc:1061] running on GCE node
I20260809 07:57:18.159081 21163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:18.160189 21163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:18.161505 21163 hybrid_clock.cc:648] HybridClock initialized: now 1786262238161446 us; error 54 us; skew 500 ppm
I20260809 07:57:18.161887 21163 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:18 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:18.165416 21163 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:18.167618 21163 webserver.cc:533] Webserver started at http://127.26.37.1:45791/ using document root <none> and password file <none>
I20260809 07:57:18.168382 21163 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:18.168546 21163 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:18.174396 21163 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:57:18.177635 21181 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:18.179167 21163 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:57:18.179342 21163 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c31f49d13b3e49f28f6113ad88b4f2f7"
format_stamp: "Formatted at 2026-08-09 07:57:16 on dist-test-slave-bw88"
I20260809 07:57:18.180053 21163 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:18.203102 21163 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:18.207684 21163 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:18.208001 21163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:18.209090 21163 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:18.210896 21163 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:18.210985 21163 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:18.211112 21163 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:18.211210 21163 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:18.248006 21163 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35495
I20260809 07:57:18.248083 21344 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35495 every 8 connection(s)
I20260809 07:57:18.249657 21163 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:18.260002 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21163
Aug 09 07:57:18 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:57:18 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:57:18 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:57:18 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:18.308979 21347 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41995
I20260809 07:57:18.309116 21349 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34577
I20260809 07:57:18.309374 21345 heartbeater.cc:344] Connected to a master server at 127.26.37.59:38289
I20260809 07:57:18.309497 21349 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:18.309494 21347 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:18.309531 21345 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:18.310873 21349 heartbeater.cc:507] Master 127.26.37.61:34577 requested a full tablet report, sending...
I20260809 07:57:18.310905 21347 heartbeater.cc:507] Master 127.26.37.62:41995 requested a full tablet report, sending...
I20260809 07:57:18.311051 21345 heartbeater.cc:507] Master 127.26.37.59:38289 requested a full tablet report, sending...
I20260809 07:57:18.311668 21346 heartbeater.cc:344] Connected to a master server at 127.26.37.60:41421
I20260809 07:57:18.311842 21346 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:18.312352 21346 heartbeater.cc:507] Master 127.26.37.60:41421 requested a full tablet report, sending...
I20260809 07:57:18.313455 20309 ts_manager.cc:194] Re-registered known tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:18.313621 20520 ts_manager.cc:194] Re-registered known tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:18.313728 21084 ts_manager.cc:194] Registered new tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:18.314693 20415 ts_manager.cc:194] Re-registered known tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:18.314806 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:57:18.314788 20520 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:39005
I20260809 07:57:18.322163 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:57:18.322337 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20267
I20260809 07:57:18.323190 20355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:18.349926 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:18.350289 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:18.351660 21362 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c: attempting to promote NON_VOTER 2ef5d7cce34142e896a4cc37e6cc22ba to VOTER
I20260809 07:57:18.352692 21362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } }
I20260809 07:57:18.355584 21097 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 1 LEARNER]: Refusing update from remote peer 56dc1770c07d45d3b3838210865c856c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:57:18.355723 20428 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 1 FOLLOWER]: Refusing update from remote peer 56dc1770c07d45d3b3838210865c856c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:57:18.356299 21370 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:57:18.356700 21362 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:57:18.360526 21370 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 2ef5d7cce34142e896a4cc37e6cc22ba (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:18.361524 21097 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 2ef5d7cce34142e896a4cc37e6cc22ba (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:18.363605 21362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:18.363873 21362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:18.364202 21362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:18.364388 21362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:18.364655 21362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:18.364393 20428 raft_consensus.cc:2955] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 2ef5d7cce34142e896a4cc37e6cc22ba (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:18.364822 21362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:18.366233 21375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:18.366506 21375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef5d7cce34142e896a4cc37e6cc22ba [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:18.368314 21376 sys_catalog.cc:455] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:18.368573 21376 sys_catalog.cc:458] T 00000000000000000000000000000000 P c859add327ef4282843e216ad200f66b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:18.439031 20326 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 1 FOLLOWER]: Refusing update from remote peer 56dc1770c07d45d3b3838210865c856c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:57:18.439568 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20267
I20260809 07:57:18.443979 21370 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 07:57:18.469511 20496 connection.cc:573] client connection to 127.26.37.62:41995 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:57:18.469813 20496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:57:18.471027 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41995
--webserver_interface=127.26.37.62
--webserver_port=39645
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:18.472257 20496 consensus_peers.cc:597] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c -> Peer d2d5a364f6f944dfaf4595b5c7d960d8 (127.26.37.62:41995): Couldn't send request to peer d2d5a364f6f944dfaf4595b5c7d960d8. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:41995: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 07:57:18.835072 21379 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:18.835481 21379 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:18.835635 21379 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:18.843917 21379 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:18.844055 21379 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:18.844197 21379 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:18.844281 21379 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:18.855155 21379 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:41995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=39645
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:18.856892 21379 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:18.858346 21379 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:18.866971 21387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:18.867046 21389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:18.868356 21391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:18.868826 21379 server_base.cc:1061] running on GCE node
I20260809 07:57:18.869694 21379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:18.871169 21379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:18.872409 21379 hybrid_clock.cc:648] HybridClock initialized: now 1786262238872370 us; error 39 us; skew 500 ppm
I20260809 07:57:18.872774 21379 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:18 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:18.875928 21379 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:57:18.877583 21379 webserver.cc:533] Webserver started at http://127.26.37.62:39645/ using document root <none> and password file <none>
I20260809 07:57:18.878376 21379 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:18.878542 21379 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:18.884042 21379 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:57:18.887080 21399 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:18.888469 21379 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:57:18.888650 21379 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "d2d5a364f6f944dfaf4595b5c7d960d8"
format_stamp: "Formatted at 2026-08-09 07:57:13 on dist-test-slave-bw88"
I20260809 07:57:18.889463 21379 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:18.910054 21379 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:18.917869 21379 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:18.918179 21379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:18.939060 21379 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:41995
I20260809 07:57:18.939091 21475 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:41995 every 8 connection(s)
I20260809 07:57:18.940845 21379 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:57:18.948464 21478 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:57:18.948836 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21379
I20260809 07:57:18.949159 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21163
I20260809 07:57:18.953197 21478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: Bootstrap starting.
I20260809 07:57:18.966904 21339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:18.975468 21478 log.cc:826] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:18.986914 21478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:57:18.987658 21478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: Bootstrap complete.
I20260809 07:57:18.996245 21478 raft_consensus.cc:359] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } }
I20260809 07:57:18.997259 21478 raft_consensus.cc:740] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d5a364f6f944dfaf4595b5c7d960d8, State: Initialized, Role: FOLLOWER
I20260809 07:57:18.998067 21478 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } }
I20260809 07:57:18.999104 21486 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: true } } }
I20260809 07:57:18.999467 21486 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:19.000268 21478 sys_catalog.cc:565] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:57:19.003000 21437 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 2ef5d7cce34142e896a4cc37e6cc22ba (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:19.011371 21505 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:19.012377 21505 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:19.012869 21486 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56dc1770c07d45d3b3838210865c856c. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:19.013059 21486 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:19.013132 21509 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262238, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:57:19.074295 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21163
I20260809 07:57:19.099576 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35495
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--webserver_port=45791
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:19.437805 21370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dc1770c07d45d3b3838210865c856c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2d5a364f6f944dfaf4595b5c7d960d8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 41995 } } peers { permanent_uuid: "c859add327ef4282843e216ad200f66b" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 34577 } } peers { permanent_uuid: "56dc1770c07d45d3b3838210865c856c" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 41421 } } peers { permanent_uuid: "2ef5d7cce34142e896a4cc37e6cc22ba" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 38289 } attrs { promote: false } } }
I20260809 07:57:19.438145 21370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dc1770c07d45d3b3838210865c856c [sys.catalog]: This master's current role is: LEADER
W20260809 07:57:19.467221 21514 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:19.467562 21514 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:19.467666 21514 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:19.476382 21514 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:19.476626 21514 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:19.488487 21514 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45791
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:19.490439 21514 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:19.491818 21514 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:19.502400 21524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:19.502406 21525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:19.503679 21528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:19.504181 21514 server_base.cc:1061] running on GCE node
I20260809 07:57:19.504702 21514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:19.505625 21514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:19.506853 21514 hybrid_clock.cc:648] HybridClock initialized: now 1786262239506801 us; error 41 us; skew 500 ppm
I20260809 07:57:19.507236 21514 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262239, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:19.510681 21514 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:19.512626 21514 webserver.cc:533] Webserver started at http://127.26.37.1:45791/ using document root <none> and password file <none>
I20260809 07:57:19.513578 21514 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:19.513711 21514 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:19.520588 21514 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 07:57:19.523566 21540 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:19.525199 21514 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:57:19.525437 21514 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c31f49d13b3e49f28f6113ad88b4f2f7"
format_stamp: "Formatted at 2026-08-09 07:57:16 on dist-test-slave-bw88"
I20260809 07:57:19.526197 21514 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:19.544512 21514 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:19.549536 21514 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:19.549815 21514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:19.550972 21514 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:19.552973 21514 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:19.553078 21514 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:19.553186 21514 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:19.553261 21514 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:19.590413 21514 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35495
I20260809 07:57:19.590466 21699 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35495 every 8 connection(s)
I20260809 07:57:19.592005 21514 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:19.597640 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21514
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262239, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262239, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262239, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:57:19 dist-test-slave-bw88 krb5kdc[19837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262239, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:19.646350 21702 heartbeater.cc:344] Connected to a master server at 127.26.37.62:41995
I20260809 07:57:19.646807 21702 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:19.647815 21702 heartbeater.cc:507] Master 127.26.37.62:41995 requested a full tablet report, sending...
I20260809 07:57:19.650612 21425 ts_manager.cc:194] Registered new tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:19.651314 21700 heartbeater.cc:344] Connected to a master server at 127.26.37.59:38289
I20260809 07:57:19.651453 21700 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:19.652046 21700 heartbeater.cc:507] Master 127.26.37.59:38289 requested a full tablet report, sending...
I20260809 07:57:19.652529 21703 heartbeater.cc:344] Connected to a master server at 127.26.37.61:34577
I20260809 07:57:19.652666 21703 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:19.653173 21703 heartbeater.cc:507] Master 127.26.37.61:34577 requested a full tablet report, sending...
I20260809 07:57:19.653537 21085 ts_manager.cc:194] Re-registered known tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:19.655318 20415 ts_manager.cc:194] Re-registered known tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:19.656041 21701 heartbeater.cc:344] Connected to a master server at 127.26.37.60:41421
I20260809 07:57:19.656176 21701 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:19.656566 21701 heartbeater.cc:507] Master 127.26.37.60:41421 requested a full tablet report, sending...
I20260809 07:57:19.657610 20520 ts_manager.cc:194] Re-registered known tserver with Master: c31f49d13b3e49f28f6113ad88b4f2f7 (127.26.37.1:35495)
I20260809 07:57:19.658459 20520 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:40989
I20260809 07:57:19.782924 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21514
W20260809 07:57:19.784457 21723 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:19.780574 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.26.37.1:35495)
0809 07:57:19.781509 (+ 935us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:19.781516 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:19.781546 (+ 30us) server_negotiation.cc:408] Connection header received
0809 07:57:19.781610 (+ 64us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:19.781617 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:19.781828 (+ 211us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:19.781987 (+ 159us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:19.782852 (+ 865us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44642: BlockingRecv error: recv got EOF from 127.0.0.1:44642 (error 108)
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":556,"threads_started":1}
W20260809 07:57:19.785326 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21514: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:35495: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:57:19.785487 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21514
I20260809 07:57:19.812366 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35495
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--webserver_port=45791
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:20.018914 21509 catalog_manager.cc:1295] Loaded cluster ID: a38988ebe5e1430ab1ffa4e6f8d2d2d5
I20260809 07:57:20.019085 21509 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: loading cluster ID for follower catalog manager: success
I20260809 07:57:20.023497 21509 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: acquiring CA information for follower catalog manager: success
I20260809 07:57:20.026257 21509 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d2d5a364f6f944dfaf4595b5c7d960d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:57:20.185689 21725 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:20.186082 21725 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:20.186277 21725 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:57:20.186374 21725 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:20.195545 21725 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:20.195811 21725 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:20.207757 21725 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45791
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:20.209722 21725 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:20.211315 21725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:20.221571 21732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:20.221560 21731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:20.223163 21734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:20.223680 21725 server_base.cc:1061] running on GCE node
I20260809 07:57:20.224406 21725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:20.225564 21725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:20.226824 21725 hybrid_clock.cc:648] HybridClock initialized: now 1786262240226766 us; error 35 us; skew 500 ppm
I20260809 07:57:20.227202 21725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:20 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262240, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:20.231607 21725 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:20.233712 21725 webserver.cc:533] Webserver started at http://127.26.37.1:45791/ using document root <none> and password file <none>
I20260809 07:57:20.234637 21725 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:20.234748 21725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:20.242113 21725 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.003s
I20260809 07:57:20.245720 21745 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:20.247435 21725 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:57:20.247614 21725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c31f49d13b3e49f28f6113ad88b4f2f7"
format_stamp: "Formatted at 2026-08-09 07:57:16 on dist-test-slave-bw88"
I20260809 07:57:20.248525 21725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:20.269383 21725 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:20.274634 21725 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:20.274964 21725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:20.276153 21725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:20.278354 21725 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:20.278468 21725 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:20.278604 21725 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:20.278678 21725 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:57:20.279098 21725 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:57:20.279181 21725 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:57:20.279256 21725 fault_injection.cc:59] injecting fault for kudu (pid 21725): FLAGS_fault_before_start (process will exit)
I20260809 07:57:20.310093 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:35495
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421
--webserver_port=45791
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:20.703630 21756 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:20.704015 21756 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:20.704128 21756 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:20.712378 21756 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:20.712551 21756 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:20.722510 21756 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:35495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=45791
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:41995,127.26.37.61:34577,127.26.37.60:41421,127.26.37.59:38289
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:20.723966 21756 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:20.725247 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:20.734618 21766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:20.734653 21763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:20.735589 21769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:20.735847 21756 server_base.cc:1061] running on GCE node
I20260809 07:57:20.736419 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:20.737373 21756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:20.738646 21756 hybrid_clock.cc:648] HybridClock initialized: now 1786262240738589 us; error 43 us; skew 500 ppm
I20260809 07:57:20.738965 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:20 dist-test-slave-bw88 krb5kdc[19837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262240, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:20.742295 21756 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:20.743858 21756 webserver.cc:533] Webserver started at http://127.26.37.1:45791/ using document root <none> and password file <none>
I20260809 07:57:20.744583 21756 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:20.744767 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:20.750907 21756 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:57:20.754199 21777 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:20.756181 21756 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260809 07:57:20.756367 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c31f49d13b3e49f28f6113ad88b4f2f7"
format_stamp: "Formatted at 2026-08-09 07:57:16 on dist-test-slave-bw88"
I20260809 07:57:20.757591 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:20.780031 21756 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:20.783835 21756 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:20.784139 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:20.785185 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:20.787061 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:20.787176 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:20.787297 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:20.787395 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:20.821609 21756 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:35495
I20260809 07:57:20.821656 21933 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:35495 every 8 connection(s)
I20260809 07:57:20.823150 21756 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:20.825743 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 21756
I20260809 07:57:20.826164 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21756
W20260809 07:57:20.850392 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21756: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:35495: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:57:20.850574 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21756
W20260809 07:57:20.872713 21755 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:20.827199 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:55403 (local address 127.26.37.62:41995)
0809 07:57:20.827408 (+ 209us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:20.827417 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:20.844891 (+ 17474us) server_negotiation.cc:408] Connection header received
0809 07:57:20.845719 (+ 828us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:20.845729 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:20.845834 (+ 105us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:20.845933 (+ 99us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:20.871875 (+ 25942us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:55403: BlockingRecv error: recv got EOF from 127.26.37.1:55403 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 07:57:20.872771 21942 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:20.826905 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:56987 (local address 127.26.37.61:34577)
0809 07:57:20.852390 (+ 25485us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:20.852395 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:20.852411 (+ 16us) server_negotiation.cc:408] Connection header received
0809 07:57:20.852478 (+ 67us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:20.852481 (+ 3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:20.852577 (+ 96us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:20.852677 (+ 100us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:20.871857 (+ 19180us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:56987: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":25389,"thread_start_us":26316,"threads_started":1}
W20260809 07:57:20.872768 21952 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:20.841442 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:55697 (local address 127.26.37.59:38289)
0809 07:57:20.853221 (+ 11779us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:20.853226 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:20.871780 (+ 18554us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:55697: BlockingRecv error: recv got EOF from 127.26.37.1:55697 (error 108)
Metrics: {"server-negotiator.queue_time_us":11683,"thread_start_us":11582,"threads_started":1}
W20260809 07:57:20.872766 21950 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:20.840936 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:55207 (local address 127.26.37.60:41421)
0809 07:57:20.846536 (+ 5600us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:20.846544 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:20.871780 (+ 25236us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:55207: BlockingRecv error: recv got EOF from 127.26.37.1:55207 (error 108)
Metrics: {"server-negotiator.queue_time_us":5460,"thread_start_us":5334,"threads_started":1}
I20260809 07:57:20.873571 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21379
W20260809 07:57:20.875763 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 21379: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:41995: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:57:20.875851 21755 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:20.874322 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.26.37.62:41995)
0809 07:57:20.874509 (+ 187us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:20.874513 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:20.874556 (+ 43us) server_negotiation.cc:408] Connection header received
0809 07:57:20.874724 (+ 168us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:20.874728 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:20.874794 (+ 66us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:20.874876 (+ 82us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:20.875630 (+ 754us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54042: BlockingRecv error: recv got EOF from 127.0.0.1:54042 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:57:20.875870 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 21379
I20260809 07:57:20.904546 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20372
I20260809 07:57:20.905993 20456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:21.019027 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20372
I20260809 07:57:21.048453 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20471
I20260809 07:57:21.049697 20562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:21.172500 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20471
I20260809 07:57:21.200196 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 20876
I20260809 07:57:21.201432 21127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:21.314399 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 20876
Shutting down hive metastore.
2026-08-09T07:57:21Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/10 (27213 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/11
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:57:21 dist-test-slave-bw88 krb5kdc[21969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:57:21 dist-test-slave-bw88 krb5kdc[21969](info): set up 2 sockets
Aug 09 07:57:21 dist-test-slave-bw88 krb5kdc[21969](info): commencing operation
krb5kdc: starting...
W20260809 07:57:23.969220 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.209s user 0.004s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:57:24 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:57:24Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:57:24Z Disabled control of system clock
2026-08-09T07:57:24Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:57:24Z Disabled control of system clock
2026-08-09T07:57:24Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:57:24Z Disabled control of system clock
2026-08-09T07:57:24Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:57:24Z Disabled control of system clock
2026-08-09T07:57:24Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:57:24Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf
2026-08-09 07:57:31: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-tasktiCCAC/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 09 07:57:33 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262253, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:57:39 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262259, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:57:39 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786262259, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:40.872270 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:43023
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:41.228442 22386 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:41.228839 22386 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:41.228999 22386 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:41.237308 22386 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:41.237475 22386 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:41.237586 22386 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:41.237668 22386 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:41.248013 22386 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:43023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:41.249675 22386 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:41.251056 22386 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:41.260281 22393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:41.261132 22392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:41.261600 22386 server_base.cc:1061] running on GCE node
W20260809 07:57:41.262277 22397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:41.263135 22386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:41.264691 22386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:41.265926 22386 hybrid_clock.cc:648] HybridClock initialized: now 1786262261265800 us; error 95 us; skew 500 ppm
I20260809 07:57:41.266295 22386 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:41 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262261, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:41.269299 22386 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:57:41.271052 22386 webserver.cc:533] Webserver started at http://127.26.37.62:46569/ using document root <none> and password file <none>
I20260809 07:57:41.271756 22386 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:41.271899 22386 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:41.272243 22386 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:41.274823 22386 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "0b91d76e03c64b0c80139683c4ff9283"
format_stamp: "Formatted at 2026-08-09 07:57:41 on dist-test-slave-bw88"
I20260809 07:57:41.275609 22386 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "0b91d76e03c64b0c80139683c4ff9283"
format_stamp: "Formatted at 2026-08-09 07:57:41 on dist-test-slave-bw88"
I20260809 07:57:41.280779 22386 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:57:41.284192 22409 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:41.285621 22386 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:41.285808 22386 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "0b91d76e03c64b0c80139683c4ff9283"
format_stamp: "Formatted at 2026-08-09 07:57:41 on dist-test-slave-bw88"
I20260809 07:57:41.285998 22386 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:41.308247 22386 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:41.311946 22386 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:41.312248 22386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:41.332765 22386 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:43023
I20260809 07:57:41.332796 22494 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:43023 every 8 connection(s)
I20260809 07:57:41.334275 22386 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:57:41.336861 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22386
I20260809 07:57:41.337093 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:57:41.339208 22496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:41.345019 22496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:41 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262261, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:41.368444 22496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:57:41.369907 22414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43433: connect: Connection refused (error 111)
W20260809 07:57:41.371615 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43433: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43433: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:41.420962 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } attempt: 1
W20260809 07:57:41.423142 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43433: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43433: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:41.484441 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:43433
--webserver_interface=127.26.37.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:41.491406 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } attempt: 2
W20260809 07:57:41.493535 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43433: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43433: connect: Connection refused (error 111)
I20260809 07:57:41.584815 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } attempt: 3
W20260809 07:57:41.587105 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43433: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43433: connect: Connection refused (error 111)
I20260809 07:57:41.730407 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } attempt: 4
W20260809 07:57:41.732585 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:43433: Network error: Client connection negotiation failed: client connection to 127.26.37.61:43433: connect: Connection refused (error 111)
W20260809 07:57:41.839483 22506 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:41.839843 22506 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:41.840014 22506 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:41.847983 22506 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:41.848102 22506 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:41.848160 22506 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:41.848197 22506 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:41.858933 22506 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:41.860675 22506 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:41.862114 22506 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:41.870452 22512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:41.870623 22515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:41.870452 22511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:41.871683 22506 server_base.cc:1061] running on GCE node
I20260809 07:57:41.872437 22506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:41.873973 22506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:41.875219 22506 hybrid_clock.cc:648] HybridClock initialized: now 1786262261875064 us; error 127 us; skew 500 ppm
I20260809 07:57:41.875635 22506 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:41 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262261, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:41.878669 22506 init.cc:377] Logged in from keytab as kudu/127.26.37.61@KRBTEST.COM (short username kudu)
I20260809 07:57:41.880260 22506 webserver.cc:533] Webserver started at http://127.26.37.61:46037/ using document root <none> and password file <none>
I20260809 07:57:41.880992 22506 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:41.881091 22506 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:41.881472 22506 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:41.883862 22506 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "7770c5974390401cafa41ab5239cdf20"
format_stamp: "Formatted at 2026-08-09 07:57:41 on dist-test-slave-bw88"
I20260809 07:57:41.884599 22506 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "7770c5974390401cafa41ab5239cdf20"
format_stamp: "Formatted at 2026-08-09 07:57:41 on dist-test-slave-bw88"
I20260809 07:57:41.889882 22506 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 07:57:41.893217 22526 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:41.894585 22506 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:41.894731 22506 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "7770c5974390401cafa41ab5239cdf20"
format_stamp: "Formatted at 2026-08-09 07:57:41 on dist-test-slave-bw88"
I20260809 07:57:41.894866 22506 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:41.913918 22506 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:41.918371 22506 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:41.918648 22506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:41.939436 22506 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:43433
I20260809 07:57:41.939483 22595 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:43433 every 8 connection(s)
I20260809 07:57:41.941010 22506 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:57:41.945262 22596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:41.950830 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22506
I20260809 07:57:41.951056 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:57:41.951342 22596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:41 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262261, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:41.975427 22596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:41 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262261, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.37.60@KRBTEST.COM; defaulting to no policy
I20260809 07:57:41.992487 22596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.60@KRBTEST.COM" created.
W20260809 07:57:41.993654 22530 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
W20260809 07:57:41.995355 22596 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:42.031870 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } attempt: 5
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262261, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:42.044720 22596 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 1
I20260809 07:57:42.045469 22496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:57:42.046783 22596 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
W20260809 07:57:42.047619 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.26.37.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.60@KRBTEST.COM" created.
I20260809 07:57:42.098836 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 1
W20260809 07:57:42.101244 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:42.107182 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:39991
--webserver_interface=127.26.37.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:42.114997 22596 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 2
W20260809 07:57:42.116959 22596 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
I20260809 07:57:42.169523 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 2
W20260809 07:57:42.171690 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
I20260809 07:57:42.208184 22596 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 3
W20260809 07:57:42.210443 22596 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
I20260809 07:57:42.277915 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 3
W20260809 07:57:42.280061 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
I20260809 07:57:42.353705 22596 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 4
W20260809 07:57:42.355818 22596 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
I20260809 07:57:42.457283 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 4
W20260809 07:57:42.459411 22496 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.60:39991: Network error: Client connection negotiation failed: client connection to 127.26.37.60:39991: connect: Connection refused (error 111)
W20260809 07:57:42.466534 22612 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:42.466853 22612 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:42.467015 22612 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:42.474926 22612 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:42.475032 22612 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:42.475154 22612 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:42.475230 22612 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:42.485702 22612 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.60:39991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.37.60
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:42.487290 22612 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:42.488783 22612 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:42.497273 22619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:42.497298 22621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:42.497859 22624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:42.498334 22612 server_base.cc:1061] running on GCE node
I20260809 07:57:42.499045 22612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:42.500540 22612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:42.503834 22612 hybrid_clock.cc:648] HybridClock initialized: now 1786262262503633 us; error 165 us; skew 500 ppm
I20260809 07:57:42.504139 22612 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:42.507287 22612 init.cc:377] Logged in from keytab as kudu/127.26.37.60@KRBTEST.COM (short username kudu)
I20260809 07:57:42.508769 22612 webserver.cc:533] Webserver started at http://127.26.37.60:36101/ using document root <none> and password file <none>
I20260809 07:57:42.509555 22612 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:42.509658 22612 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:42.509948 22612 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:42.512380 22612 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/data/instance:
uuid: "981fdec031e74ccf93b919d9882446a7"
format_stamp: "Formatted at 2026-08-09 07:57:42 on dist-test-slave-bw88"
I20260809 07:57:42.513132 22612 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/wal/instance:
uuid: "981fdec031e74ccf93b919d9882446a7"
format_stamp: "Formatted at 2026-08-09 07:57:42 on dist-test-slave-bw88"
I20260809 07:57:42.518514 22612 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 07:57:42.521785 22637 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:42.523079 22612 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 07:57:42.523264 22612 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/wal
uuid: "981fdec031e74ccf93b919d9882446a7"
format_stamp: "Formatted at 2026-08-09 07:57:42 on dist-test-slave-bw88"
I20260809 07:57:42.523458 22612 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:42.541396 22612 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:42.545087 22612 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:42.545425 22612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:42.566272 22612 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.60:39991
I20260809 07:57:42.566321 22705 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.60:39991 every 8 connection(s)
I20260809 07:57:42.567852 22612 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/data/info.pb
I20260809 07:57:42.572150 22707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:42.572543 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22612
I20260809 07:57:42.572752 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-2/wal/instance
I20260809 07:57:42.581802 22707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:42.607817 22707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:57:42.623222 22707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:57:42.642699 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: Bootstrap starting.
I20260809 07:57:42.646135 22707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:42.647259 22707 log.cc:826] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:42.650244 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: No bootstrap required, opened a new log
I20260809 07:57:42.655083 22596 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 5
I20260809 07:57:42.656059 22707 raft_consensus.cc:359] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.656468 22707 raft_consensus.cc:385] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:42.656606 22707 raft_consensus.cc:740] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981fdec031e74ccf93b919d9882446a7, State: Initialized, Role: FOLLOWER
I20260809 07:57:42.657294 22707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.658551 22719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.658872 22719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:42.659706 22707 sys_catalog.cc:565] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262261, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.61@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
W20260809 07:57:42.676265 22730 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:57:42.676398 22730 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:57:42.677206 22596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: Bootstrap starting.
I20260809 07:57:42.668958 22612 master_runner.cc:187] Error getting master registration for 127.26.37.62:43023: OK, instance_id { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" instance_seqno: 1786262261327306 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:57:42.679924 22612 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:57:42.681531 22596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:42.682955 22596 log.cc:826] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:42.686049 22596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: No bootstrap required, opened a new log
I20260809 07:57:42.693626 22596 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.693970 22596 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:42.694103 22596 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7770c5974390401cafa41ab5239cdf20, State: Initialized, Role: FOLLOWER
I20260809 07:57:42.694823 22596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.696630 22733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.697024 22733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:42.697996 22596 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:57:42.708304 22747 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:57:42.708489 22747 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:57:42.705327 22506 master_runner.cc:187] Error getting master registration for 127.26.37.62:43023: OK, instance_id { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" instance_seqno: 1786262261327306 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:57:42.712877 22506 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:57:42.736688 22496 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } attempt: 5
Aug 09 07:57:42 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262261, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:57:42.755157 22496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: Bootstrap starting.
I20260809 07:57:42.760291 22496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:42.761741 22496 log.cc:826] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:42.764743 22496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: No bootstrap required, opened a new log
I20260809 07:57:42.770243 22496 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.770540 22496 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:42.770690 22496 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b91d76e03c64b0c80139683c4ff9283, State: Initialized, Role: FOLLOWER
I20260809 07:57:42.771400 22496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.772444 22752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.772835 22752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:42.773726 22496 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:57:42.794461 22386 master_runner.cc:264] No leader master found from master 0b91d76e03c64b0c80139683c4ff9283
I20260809 07:57:42.794646 22386 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:57:42.797518 22644 tablet.cc:2406] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:57:42.798521 22769 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:57:42.798667 22769 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 07:57:42.815186 22422 tablet.cc:2406] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:57:42.839488 22719 raft_consensus.cc:493] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:57:42.839804 22719 raft_consensus.cc:515] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.843937 22719 leader_election.cc:290] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b91d76e03c64b0c80139683c4ff9283 (127.26.37.62:43023), 7770c5974390401cafa41ab5239cdf20 (127.26.37.61:43433)
I20260809 07:57:42.845309 22456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "981fdec031e74ccf93b919d9882446a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b91d76e03c64b0c80139683c4ff9283" is_pre_election: true
I20260809 07:57:42.845865 22456 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 981fdec031e74ccf93b919d9882446a7 in term 0.
I20260809 07:57:42.846942 22559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "981fdec031e74ccf93b919d9882446a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7770c5974390401cafa41ab5239cdf20" is_pre_election: true
I20260809 07:57:42.847429 22559 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 981fdec031e74ccf93b919d9882446a7 in term 0.
I20260809 07:57:42.848045 22641 leader_election.cc:304] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b91d76e03c64b0c80139683c4ff9283, 981fdec031e74ccf93b919d9882446a7; no voters:
I20260809 07:57:42.849079 22719 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:57:42.849236 22719 raft_consensus.cc:493] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:57:42.849350 22719 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:57:42.852792 22719 raft_consensus.cc:515] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.854046 22456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "981fdec031e74ccf93b919d9882446a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b91d76e03c64b0c80139683c4ff9283"
I20260809 07:57:42.854241 22456 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:57:42.858120 22456 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 981fdec031e74ccf93b919d9882446a7 in term 1.
I20260809 07:57:42.858652 22641 leader_election.cc:304] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b91d76e03c64b0c80139683c4ff9283, 981fdec031e74ccf93b919d9882446a7; no voters:
I20260809 07:57:42.866145 22559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "981fdec031e74ccf93b919d9882446a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7770c5974390401cafa41ab5239cdf20"
I20260809 07:57:42.866389 22559 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:57:42.870702 22559 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 981fdec031e74ccf93b919d9882446a7 in term 1.
I20260809 07:57:42.871508 22719 leader_election.cc:290] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [CANDIDATE]: Term 1 election: Requested vote from peers 0b91d76e03c64b0c80139683c4ff9283 (127.26.37.62:43023), 7770c5974390401cafa41ab5239cdf20 (127.26.37.61:43433)
I20260809 07:57:42.871992 22719 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:57:42.872498 22719 raft_consensus.cc:697] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 1 LEADER]: Becoming Leader. State: Replica: 981fdec031e74ccf93b919d9882446a7, State: Running, Role: LEADER
I20260809 07:57:42.873276 22719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } }
I20260809 07:57:42.892184 22771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 981fdec031e74ccf93b919d9882446a7. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.892653 22771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:42.899631 22774 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:57:42.902364 22774 catalog_manager.cc:1520] Initializing Kudu cluster ID...
W20260809 07:57:42.921593 22535 tablet.cc:2406] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:57:42.922492 22559 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 1 FOLLOWER]: Refusing update from remote peer 981fdec031e74ccf93b919d9882446a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:57:42.923612 22771 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:57:42.931860 22456 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 1 FOLLOWER]: Refusing update from remote peer 981fdec031e74ccf93b919d9882446a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:57:42.933018 22771 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:57:42.956480 22733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 981fdec031e74ccf93b919d9882446a7. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.956772 22733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:42.970360 22775 mvcc.cc:204] Tried to move back new op lower bound from 7316530228917342208 to 7316530228742995968. Current Snapshot: MvccSnapshot[applied={T|T < 7316530228917342208}]
I20260809 07:57:42.973806 22719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.974104 22719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:42.976321 22733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.976532 22733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:42.979058 22774 catalog_manager.cc:1383] Generated new cluster ID: 0057718651184977a6dea330450080b7
I20260809 07:57:42.979159 22774 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:57:42.990871 22752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 981fdec031e74ccf93b919d9882446a7. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.991180 22752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:42.994074 22771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:42.994318 22771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:42.996248 22777 mvcc.cc:204] Tried to move back new op lower bound from 7316530228917342208 to 7316530228742995968. Current Snapshot: MvccSnapshot[applied={T|T < 7316530228917342208}]
I20260809 07:57:43.005745 22752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } }
I20260809 07:57:43.005996 22752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:43.023183 22774 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:57:43.025031 22774 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:57:43.058100 22774 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: Generated new TSK 0
I20260809 07:57:43.059521 22774 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
W20260809 07:57:43.216549 26772 mini_kdc.cc:268] Time spent creating service keytab for kudu/127.26.37.1: real 0.139s user 0.005s sys 0.031s
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
W20260809 07:57:43.320384 26772 mini_kdc.cc:268] Time spent creating service keytab for HTTP/127.26.37.1: real 0.104s user 0.000s sys 0.035s
I20260809 07:57:43.321640 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:43.710860 22747 catalog_manager.cc:1295] Loaded cluster ID: 0057718651184977a6dea330450080b7
I20260809 07:57:43.710956 22747 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: loading cluster ID for follower catalog manager: success
I20260809 07:57:43.713553 22747 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: acquiring CA information for follower catalog manager: success
I20260809 07:57:43.715309 22747 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:57:43.719964 22793 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:43.720300 22793 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:43.720471 22793 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:43.729950 22793 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:43.730185 22793 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:43.741568 22793 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:43.743193 22793 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:43.744585 22793 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:43.754621 22801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:43.754730 22800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:43.754967 22806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:43.756435 22793 server_base.cc:1061] running on GCE node
I20260809 07:57:43.756946 22793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:43.758018 22793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:43.759297 22793 hybrid_clock.cc:648] HybridClock initialized: now 1786262263759148 us; error 116 us; skew 500 ppm
I20260809 07:57:43.759645 22793 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:43 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262263, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:43.763017 22793 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:43.764540 22793 webserver.cc:533] Webserver started at http://127.26.37.1:33301/ using document root <none> and password file <none>
I20260809 07:57:43.765259 22793 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:43.765420 22793 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:43.765779 22793 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:43.768088 22793 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "c2e5ce4111f54d199ab7cbefde201f62"
format_stamp: "Formatted at 2026-08-09 07:57:43 on dist-test-slave-bw88"
I20260809 07:57:43.768916 22793 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "c2e5ce4111f54d199ab7cbefde201f62"
format_stamp: "Formatted at 2026-08-09 07:57:43 on dist-test-slave-bw88"
I20260809 07:57:43.774281 22793 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:57:43.777665 22815 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:43.778949 22793 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:57:43.779114 22793 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c2e5ce4111f54d199ab7cbefde201f62"
format_stamp: "Formatted at 2026-08-09 07:57:43 on dist-test-slave-bw88"
I20260809 07:57:43.779292 22793 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:43.797474 22793 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:43.800693 22769 catalog_manager.cc:1295] Loaded cluster ID: 0057718651184977a6dea330450080b7
I20260809 07:57:43.800770 22769 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: loading cluster ID for follower catalog manager: success
I20260809 07:57:43.801759 22793 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:43.802054 22793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:43.803030 22793 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:43.803717 22769 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: acquiring CA information for follower catalog manager: success
I20260809 07:57:43.804600 22793 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:43.804701 22793 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:43.804836 22793 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:43.804920 22793 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:43.805912 22769 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:57:43.839416 22793 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46463
I20260809 07:57:43.839548 22964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46463 every 8 connection(s)
I20260809 07:57:43.840859 22793 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:43.844019 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22793
I20260809 07:57:43.844238 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:57:43 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262263, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:57:43 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262263, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:57:43 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262263, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:43.868929 22965 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39991
I20260809 07:57:43.869372 22965 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:43.870573 22965 heartbeater.cc:507] Master 127.26.37.60:39991 requested a full tablet report, sending...
I20260809 07:57:43.871520 22967 heartbeater.cc:344] Connected to a master server at 127.26.37.62:43023
I20260809 07:57:43.871673 22967 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:43.871912 22968 heartbeater.cc:344] Connected to a master server at 127.26.37.61:43433
I20260809 07:57:43.872069 22968 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:43.872313 22967 heartbeater.cc:507] Master 127.26.37.62:43023 requested a full tablet report, sending...
I20260809 07:57:43.872553 22968 heartbeater.cc:507] Master 127.26.37.61:43433 requested a full tablet report, sending...
I20260809 07:57:43.873281 22660 ts_manager.cc:194] Registered new tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:43.874306 22546 ts_manager.cc:194] Registered new tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:43.875313 22660 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:41467
I20260809 07:57:43.876557 22442 ts_manager.cc:194] Registered new tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:43.890290 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.59@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.59@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.59 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:44.051090 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:36615
--webserver_interface=127.26.37.59
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:44.410506 22986 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:44.410843 22986 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:44.410976 22986 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:44.419221 22986 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:44.419324 22986 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:44.419381 22986 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:44.419456 22986 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:44.430435 22986 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.59
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.59:36615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.26.37.59
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:44.432271 22986 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:44.433831 22986 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:44.442835 22994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:44.443002 22996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:44.442835 22993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:44.444595 22986 server_base.cc:1061] running on GCE node
I20260809 07:57:44.445482 22986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:44.447029 22986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:44.448276 22986 hybrid_clock.cc:648] HybridClock initialized: now 1786262264448134 us; error 111 us; skew 500 ppm
I20260809 07:57:44.448637 22986 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262264, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:44.452368 22986 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:57:44.454362 22986 webserver.cc:533] Webserver started at http://127.26.37.59:42101/ using document root <none> and password file <none>
I20260809 07:57:44.455206 22986 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:44.455382 22986 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:44.455806 22986 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:44.458745 22986 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data/instance:
uuid: "6c0948d09bce4b0aafa0ee019e6bbc33"
format_stamp: "Formatted at 2026-08-09 07:57:44 on dist-test-slave-bw88"
I20260809 07:57:44.459594 22986 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal/instance:
uuid: "6c0948d09bce4b0aafa0ee019e6bbc33"
format_stamp: "Formatted at 2026-08-09 07:57:44 on dist-test-slave-bw88"
I20260809 07:57:44.465795 22986 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260809 07:57:44.469504 23003 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:44.470912 22986 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:44.471103 22986 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "6c0948d09bce4b0aafa0ee019e6bbc33"
format_stamp: "Formatted at 2026-08-09 07:57:44 on dist-test-slave-bw88"
I20260809 07:57:44.471305 22986 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:44.493774 22986 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:44.497840 22986 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:44.498145 22986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:44.518453 22986 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:36615
I20260809 07:57:44.518525 23069 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:36615 every 8 connection(s)
I20260809 07:57:44.520128 22986 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:57:44.524348 23070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:44.529923 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 22986
I20260809 07:57:44.530247 23070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262264, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:57:44.551899 23070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262264, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:44.564827 23070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262264, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
I20260809 07:57:44.583789 23070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262264, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:57:44.599969 23070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Bootstrap starting.
I20260809 07:57:44.603228 23070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:44.604549 23070 log.cc:826] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:44.607046 23070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: No bootstrap required, opened a new log
I20260809 07:57:44.611902 23070 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } }
I20260809 07:57:44.612250 23070 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:44.612365 23070 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0948d09bce4b0aafa0ee019e6bbc33, State: Initialized, Role: FOLLOWER
I20260809 07:57:44.613034 23070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } }
I20260809 07:57:44.613974 23082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } } }
I20260809 07:57:44.614230 23082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:44.614907 23070 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:57:44.623791 23100 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:57:44.623921 23100 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:57:44.631946 22986 master_runner.cc:429] Detected that this master 6c0948d09bce4b0aafa0ee019e6bbc33 is joining an existing cluster
I20260809 07:57:44.632095 22986 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260809 07:57:44.633013 22986 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:57:44.633410 22986 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:57:44.633607 22986 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: stopping tablet replica
I20260809 07:57:44.635125 22986 master_runner.cc:434] Initiating AddMaster RPC to add 127.26.37.59:36615
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262262, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.60@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:57:44.690793 22660 catalog_manager.cc:7224] Initiating ChangeConfig request to add master 127.26.37.59:36615: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33"
member_type: NON_VOTER
last_known_addr {
host: "127.26.37.59"
port: 36615
}
attrs {
promote: true
}
}
dest_uuid: "981fdec031e74ccf93b919d9882446a7"
cas_config_opid_index: -1
I20260809 07:57:44.692240 22660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } }
I20260809 07:57:44.695163 22454 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 1 FOLLOWER]: Refusing update from remote peer 981fdec031e74ccf93b919d9882446a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:57:44.695818 23114 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260809 07:57:44.696795 22639 consensus_peers.cc:597] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 -> Peer 6c0948d09bce4b0aafa0ee019e6bbc33 (127.26.37.59:36615): Couldn't send request to peer 6c0948d09bce4b0aafa0ee019e6bbc33. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:57:44.697685 22558 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 1 FOLLOWER]: Refusing update from remote peer 981fdec031e74ccf93b919d9882446a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:57:44.698215 23114 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:57:44.700732 22771 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6c0948d09bce4b0aafa0ee019e6bbc33 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.702425 22454 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6c0948d09bce4b0aafa0ee019e6bbc33 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.704238 22771 catalog_manager.cc:7228] Successfully completed master ChangeConfig request to add master 127.26.37.59:36615
I20260809 07:57:44.705040 22986 master.cc:562] Master@127.26.37.59:36615 shutting down...
I20260809 07:57:44.707046 22771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.707252 22771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:44.706666 22558 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6c0948d09bce4b0aafa0ee019e6bbc33 (127.26.37.59) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.706947 23114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.707517 23114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:44.707810 23114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.707995 23114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:44.716235 23121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.716653 23121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:44.720295 23120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.720628 23120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:57:44.720692 21964 connection.cc:573] client connection to 127.26.37.59:36615 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:57:44.734185 22986 master.cc:584] Master@127.26.37.59:36615 shutdown complete.
I20260809 07:57:44.734261 22986 master_runner.cc:306] Clearing existing system tablet
I20260809 07:57:44.734663 22986 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:44.734766 22986 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:44.737021 22986 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:57:44.738612 23123 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:44.738992 22986 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:44.739233 22986 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 07:57:44.739339 22986 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "6c0948d09bce4b0aafa0ee019e6bbc33"
format_stamp: "Formatted at 2026-08-09 07:57:44 on dist-test-slave-bw88"
I20260809 07:57:44.744107 22986 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:57:44.751153 22986 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:57:44.751327 22986 log.cc:1199] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260809 07:57:44.751789 22986 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Deleting consensus metadata
I20260809 07:57:44.752105 22986 master_runner.cc:316] Copying system tablet from 127.26.37.60:39991
I20260809 07:57:44.754999 22986 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: tablet copy: Beginning tablet copy session from remote peer at address 127.26.37.60:39991
I20260809 07:57:44.768452 22690 tablet_copy_service.cc:140] P 981fdec031e74ccf93b919d9882446a7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6c0948d09bce4b0aafa0ee019e6bbc33 ({username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:50834)
I20260809 07:57:44.768774 22690 tablet_copy_service.cc:161] P 981fdec031e74ccf93b919d9882446a7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6c0948d09bce4b0aafa0ee019e6bbc33 at {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:50834: session id = 6c0948d09bce4b0aafa0ee019e6bbc33-00000000000000000000000000000000
I20260809 07:57:44.773073 22690 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:57:44.775306 22986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:57:44.780261 22986 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: tablet copy: Starting download of 0 data blocks...
I20260809 07:57:44.780561 22986 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: tablet copy: Starting download of 1 WAL segments...
I20260809 07:57:44.783210 22986 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:57:44.786473 22690 tablet_copy_service.cc:342] P 981fdec031e74ccf93b919d9882446a7: Request end of tablet copy session 6c0948d09bce4b0aafa0ee019e6bbc33-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.26.37.59@KRBTEST.COM'} at 127.0.0.1:50834
I20260809 07:57:44.786680 22690 tablet_copy_service.cc:434] P 981fdec031e74ccf93b919d9882446a7: ending tablet copy session 6c0948d09bce4b0aafa0ee019e6bbc33-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6c0948d09bce4b0aafa0ee019e6bbc33
W20260809 07:57:44.794435 22986 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260809 07:57:44.797746 22986 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:44.800346 23130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:44.800895 23132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:44.806619 23134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:44.807133 22986 server_base.cc:1061] running on GCE node
I20260809 07:57:44.807433 22986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:44.808233 22986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:44.809427 22986 hybrid_clock.cc:648] HybridClock initialized: now 1786262264809331 us; error 74 us; skew 500 ppm
I20260809 07:57:44.809677 22986 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262264, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.59@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:44.811604 22986 init.cc:377] Logged in from keytab as kudu/127.26.37.59@KRBTEST.COM (short username kudu)
I20260809 07:57:44.812817 22986 webserver.cc:533] Webserver started at http://127.26.37.59:41019/ using document root <none> and password file <none>
I20260809 07:57:44.813150 22986 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:44.813254 22986 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:44.815660 22986 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:57:44.817405 23142 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:44.817847 22986 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 07:57:44.817989 22986 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
uuid: "6c0948d09bce4b0aafa0ee019e6bbc33"
format_stamp: "Formatted at 2026-08-09 07:57:44 on dist-test-slave-bw88"
I20260809 07:57:44.818177 22986 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:44.840130 22986 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:44.843680 22986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:44.862771 22986 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.59:36615
I20260809 07:57:44.862831 23211 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.59:36615 every 8 connection(s)
I20260809 07:57:44.863849 22986 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-3/data/info.pb
I20260809 07:57:44.867637 23212 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:57:44.870296 23212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Bootstrap starting.
I20260809 07:57:44.878417 22965 heartbeater.cc:499] Master 127.26.37.60:39991 was elected leader, sending a full tablet report...
I20260809 07:57:44.890046 23212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:57:44.890547 23212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: Bootstrap complete.
I20260809 07:57:44.891556 23212 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } }
I20260809 07:57:44.891799 23212 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c0948d09bce4b0aafa0ee019e6bbc33, State: Initialized, Role: LEARNER
I20260809 07:57:44.892063 23212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } }
I20260809 07:57:44.892910 23220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:44.893201 23212 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:57:44.893278 23220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: This master's current role is: LEARNER
I20260809 07:57:44.902627 23234 catalog_manager.cc:1295] Loaded cluster ID: 0057718651184977a6dea330450080b7
I20260809 07:57:44.902750 23234 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: loading cluster ID for follower catalog manager: success
I20260809 07:57:44.905534 23234 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: acquiring CA information for follower catalog manager: success
I20260809 07:57:44.907537 23234 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:57:44.919456 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22793
Aug 09 07:57:44 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:57:44.937453 22960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:45.033082 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22793
I20260809 07:57:45.056707 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46463
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--webserver_port=33301
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:45.275964 23176 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260809 07:57:45.276625 23220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 981fdec031e74ccf93b919d9882446a7. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:45.276949 23220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: This master's current role is: LEARNER
W20260809 07:57:45.426191 23240 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:45.426676 23240 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:45.426826 23240 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:45.435281 23240 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:45.435525 23240 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:45.446813 23240 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33301
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:45.448514 23240 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:45.450019 23240 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:45.460542 23249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:45.460440 23247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:45.460440 23251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:45.461866 23240 server_base.cc:1061] running on GCE node
I20260809 07:57:45.462452 23240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:45.463651 23240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:45.464923 23240 hybrid_clock.cc:648] HybridClock initialized: now 1786262265464791 us; error 99 us; skew 500 ppm
I20260809 07:57:45.465211 23240 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:45 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262265, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:45.468603 23240 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:45.470347 23240 webserver.cc:533] Webserver started at http://127.26.37.1:33301/ using document root <none> and password file <none>
I20260809 07:57:45.471112 23240 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:45.471266 23240 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:45.477097 23240 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:57:45.480028 23261 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:45.481509 23240 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:45.481696 23240 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c2e5ce4111f54d199ab7cbefde201f62"
format_stamp: "Formatted at 2026-08-09 07:57:43 on dist-test-slave-bw88"
I20260809 07:57:45.482460 23240 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:45.511426 23240 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:45.515883 23240 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:45.516198 23240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:45.517431 23240 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:45.519177 23240 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:45.519388 23240 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:45.519577 23240 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:45.519680 23240 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:45.555524 23240 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46463
I20260809 07:57:45.555626 23400 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46463 every 8 connection(s)
I20260809 07:57:45.557148 23240 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:45.558068 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 23240
Aug 09 07:57:45 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262265, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:57:45 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262265, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
Aug 09 07:57:45 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262265, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
I20260809 07:57:45.621985 23402 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39991
I20260809 07:57:45.622402 23402 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:45.623517 23402 heartbeater.cc:507] Master 127.26.37.60:39991 requested a full tablet report, sending...
I20260809 07:57:45.625737 22660 ts_manager.cc:194] Re-registered known tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:45.626121 23401 heartbeater.cc:344] Connected to a master server at 127.26.37.59:36615
I20260809 07:57:45.626255 23401 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:45.626746 23401 heartbeater.cc:507] Master 127.26.37.59:36615 requested a full tablet report, sending...
I20260809 07:57:45.626901 22660 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:51383
I20260809 07:57:45.626909 23406 heartbeater.cc:344] Connected to a master server at 127.26.37.61:43433
I20260809 07:57:45.627074 23406 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:45.627952 23406 heartbeater.cc:507] Master 127.26.37.61:43433 requested a full tablet report, sending...
I20260809 07:57:45.628944 23163 ts_manager.cc:194] Registered new tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:45.629330 22546 ts_manager.cc:194] Re-registered known tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
Aug 09 07:57:45 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262265, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:45.636287 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:57:45.638522 23404 heartbeater.cc:344] Connected to a master server at 127.26.37.62:43023
I20260809 07:57:45.638681 23404 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:45.639086 23404 heartbeater.cc:507] Master 127.26.37.62:43023 requested a full tablet report, sending...
I20260809 07:57:45.640473 22442 ts_manager.cc:194] Re-registered known tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:45.644276 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260809 07:57:45.644467 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22386
I20260809 07:57:45.645247 22489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:45.756771 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22386
I20260809 07:57:45.786945 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:43023
--webserver_interface=127.26.37.62
--webserver_port=46569
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:57:45.837700 22771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:45.838023 22771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:45.839335 22771 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7: attempting to promote NON_VOTER 6c0948d09bce4b0aafa0ee019e6bbc33 to VOTER
I20260809 07:57:45.840335 22771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } }
I20260809 07:57:45.843173 22558 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 1 FOLLOWER]: Refusing update from remote peer 981fdec031e74ccf93b919d9882446a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:57:45.843284 23176 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 1 LEARNER]: Refusing update from remote peer 981fdec031e74ccf93b919d9882446a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:57:45.843954 23423 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:57:45.844424 23422 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 07:57:45.845386 22641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.62:43023: connect: Connection refused (error 111)
I20260809 07:57:45.860966 23422 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6c0948d09bce4b0aafa0ee019e6bbc33 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:45.861784 22559 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6c0948d09bce4b0aafa0ee019e6bbc33 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
W20260809 07:57:45.865240 22641 consensus_peers.cc:597] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 -> Peer 0b91d76e03c64b0c80139683c4ff9283 (127.26.37.62:43023): Couldn't send request to peer 0b91d76e03c64b0c80139683c4ff9283. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:43023: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:57:45.866375 23422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:45.868778 23422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:45.868518 23428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:45.869072 23428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:45.868139 22771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:45.869289 22771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:45.870294 23176 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6c0948d09bce4b0aafa0ee019e6bbc33 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:45.885838 23439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:45.885212 23437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:45.886292 23439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0948d09bce4b0aafa0ee019e6bbc33 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:45.886408 23437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7770c5974390401cafa41ab5239cdf20 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:57:46.176373 23419 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:46.176709 23419 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:46.176796 23419 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:46.185133 23419 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:46.185230 23419 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:46.185289 23419 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:46.185381 23419 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:46.196439 23419 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:43023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=46569
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:46.198067 23419 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:46.199486 23419 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:46.208273 23447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:46.208953 23451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:46.209091 23445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:46.209725 23419 server_base.cc:1061] running on GCE node
I20260809 07:57:46.210533 23419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:46.212102 23419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:46.213302 23419 hybrid_clock.cc:648] HybridClock initialized: now 1786262266213147 us; error 124 us; skew 500 ppm
I20260809 07:57:46.213716 23419 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:46 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262266, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:46.217080 23419 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:57:46.218796 23419 webserver.cc:533] Webserver started at http://127.26.37.62:46569/ using document root <none> and password file <none>
I20260809 07:57:46.219563 23419 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:46.219770 23419 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:46.225274 23419 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:57:46.228020 23459 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:46.229432 23419 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:57:46.229619 23419 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "0b91d76e03c64b0c80139683c4ff9283"
format_stamp: "Formatted at 2026-08-09 07:57:41 on dist-test-slave-bw88"
I20260809 07:57:46.230360 23419 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:46.254839 23419 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:46.258860 23419 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:46.259147 23419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:46.280992 23419 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:43023
I20260809 07:57:46.281037 23523 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:43023 every 8 connection(s)
I20260809 07:57:46.282723 23419 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:57:46.285391 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 23419
I20260809 07:57:46.285779 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23240
I20260809 07:57:46.290383 23527 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:57:46.294899 23527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: Bootstrap starting.
I20260809 07:57:46.305449 23394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:46.315439 23527 log.cc:826] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:46.328089 23527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:57:46.328670 23527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: Bootstrap complete.
I20260809 07:57:46.336337 23527 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } }
I20260809 07:57:46.337052 23527 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b91d76e03c64b0c80139683c4ff9283, State: Initialized, Role: FOLLOWER
I20260809 07:57:46.337785 23527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } }
I20260809 07:57:46.338843 23539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: NON_VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: true } } }
I20260809 07:57:46.339222 23539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:46.340009 23527 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:57:46.342623 23494 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 6c0948d09bce4b0aafa0ee019e6bbc33 (127.26.37.59) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:46.349777 23549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:46.350301 23549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:46.350445 23539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 981fdec031e74ccf93b919d9882446a7. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:46.350657 23539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:57:46.367913 23560 catalog_manager.cc:1295] Loaded cluster ID: 0057718651184977a6dea330450080b7
I20260809 07:57:46.368018 23560 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: loading cluster ID for follower catalog manager: success
I20260809 07:57:46.372267 23560 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: acquiring CA information for follower catalog manager: success
I20260809 07:57:46.374854 23560 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0b91d76e03c64b0c80139683c4ff9283: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 09 07:57:46 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262266, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:46.424935 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 23240
I20260809 07:57:46.457127 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46463
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--webserver_port=33301
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:46.820947 23569 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:46.821282 23569 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:46.821465 23569 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:46.829530 23569 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:46.829764 23569 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:46.840190 23569 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33301
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:46.841899 23569 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:46.843374 23569 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:46.853816 23578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:46.854169 23580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:46.854933 23576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:46.855525 23569 server_base.cc:1061] running on GCE node
I20260809 07:57:46.856096 23569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:46.857147 23569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:46.858345 23569 hybrid_clock.cc:648] HybridClock initialized: now 1786262266858190 us; error 123 us; skew 500 ppm
I20260809 07:57:46.858652 23569 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:57:46.860421 23422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981fdec031e74ccf93b919d9882446a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0b91d76e03c64b0c80139683c4ff9283" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43023 } } peers { permanent_uuid: "7770c5974390401cafa41ab5239cdf20" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 43433 } } peers { permanent_uuid: "981fdec031e74ccf93b919d9882446a7" member_type: VOTER last_known_addr { host: "127.26.37.60" port: 39991 } } peers { permanent_uuid: "6c0948d09bce4b0aafa0ee019e6bbc33" member_type: VOTER last_known_addr { host: "127.26.37.59" port: 36615 } attrs { promote: false } } }
I20260809 07:57:46.860764 23422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 [sys.catalog]: This master's current role is: LEADER
Aug 09 07:57:46 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262266, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:46.862867 23569 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:46.875216 23569 webserver.cc:533] Webserver started at http://127.26.37.1:33301/ using document root <none> and password file <none>
I20260809 07:57:46.876044 23569 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:46.876195 23569 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:46.882048 23569 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:57:46.884841 23590 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:46.886276 23569 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:46.886451 23569 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c2e5ce4111f54d199ab7cbefde201f62"
format_stamp: "Formatted at 2026-08-09 07:57:43 on dist-test-slave-bw88"
I20260809 07:57:46.887246 23569 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:46.906626 23569 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:46.910529 23569 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:46.910817 23569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:46.911788 23569 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:46.913470 23569 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:46.913604 23569 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:46.913749 23569 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:46.913807 23569 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:46.948402 23569 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46463
I20260809 07:57:46.948493 23732 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46463 every 8 connection(s)
I20260809 07:57:46.950032 23569 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:46.954988 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 23569
Aug 09 07:57:46 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 09 07:57:46 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262266, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.61@KRBTEST.COM
Aug 09 07:57:46 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262266, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.60@KRBTEST.COM
Aug 09 07:57:46 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262266, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.59@KRBTEST.COM
I20260809 07:57:47.000289 23735 heartbeater.cc:344] Connected to a master server at 127.26.37.60:39991
I20260809 07:57:47.000572 23738 heartbeater.cc:344] Connected to a master server at 127.26.37.61:43433
I20260809 07:57:47.000787 23738 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:47.000790 23735 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:47.002249 23735 heartbeater.cc:507] Master 127.26.37.60:39991 requested a full tablet report, sending...
I20260809 07:57:47.003386 23738 heartbeater.cc:507] Master 127.26.37.61:43433 requested a full tablet report, sending...
I20260809 07:57:47.004693 22660 ts_manager.cc:194] Re-registered known tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:47.004696 22546 ts_manager.cc:194] Re-registered known tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:47.005635 23734 heartbeater.cc:344] Connected to a master server at 127.26.37.59:36615
I20260809 07:57:47.005777 23734 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:47.006093 22660 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:60199
I20260809 07:57:47.006331 23734 heartbeater.cc:507] Master 127.26.37.59:36615 requested a full tablet report, sending...
I20260809 07:57:47.007877 23163 ts_manager.cc:194] Re-registered known tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
Aug 09 07:57:47 dist-test-slave-bw88 krb5kdc[21969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262266, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:47.017267 23737 heartbeater.cc:344] Connected to a master server at 127.26.37.62:43023
I20260809 07:57:47.017488 23737 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:47.017848 23737 heartbeater.cc:507] Master 127.26.37.62:43023 requested a full tablet report, sending...
I20260809 07:57:47.019927 23480 ts_manager.cc:194] Registered new tserver with Master: c2e5ce4111f54d199ab7cbefde201f62 (127.26.37.1:46463)
I20260809 07:57:47.137282 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23569
W20260809 07:57:47.138816 23758 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:47.134959 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.26.37.1:46463)
0809 07:57:47.135646 (+ 687us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:47.135654 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:47.135691 (+ 37us) server_negotiation.cc:408] Connection header received
0809 07:57:47.135761 (+ 70us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:47.135768 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:47.135979 (+ 211us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:47.136149 (+ 170us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:47.137235 (+ 1086us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47088: BlockingRecv error: recv got EOF from 127.0.0.1:47088 (error 108)
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":374,"threads_started":1}
W20260809 07:57:47.140007 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23569: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:46463: server requires authentication, but client does not have Kerberos credentials available
W20260809 07:57:47.140054 23758 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:47.138205 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.26.37.1:46463)
0809 07:57:47.139098 (+ 893us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:47.139104 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:47.139122 (+ 18us) server_negotiation.cc:408] Connection header received
0809 07:57:47.139170 (+ 48us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:47.139175 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:47.139234 (+ 59us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:47.139313 (+ 79us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:47.139911 (+ 598us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47098: BlockingRecv error: recv got EOF from 127.0.0.1:47098 (error 108)
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":705,"threads_started":1}
I20260809 07:57:47.140131 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 23569
I20260809 07:57:47.172518 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46463
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--webserver_port=33301
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:47.536337 23760 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:47.536723 23760 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:47.536883 23760 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260809 07:57:47.537005 23760 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:47.545578 23760 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:47.545800 23760 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:47.556634 23760 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33301
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:47.558306 23760 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:47.559710 23760 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:47.570691 23767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:47.571591 23766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:47.571940 23770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:47.573591 23760 server_base.cc:1061] running on GCE node
I20260809 07:57:47.574251 23760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:47.575659 23760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:47.576994 23760 hybrid_clock.cc:648] HybridClock initialized: now 1786262267576821 us; error 116 us; skew 500 ppm
I20260809 07:57:47.577327 23760 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:47 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262267, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:47.580862 23760 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:47.582639 23760 webserver.cc:533] Webserver started at http://127.26.37.1:33301/ using document root <none> and password file <none>
I20260809 07:57:47.583408 23760 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:47.583534 23760 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:47.589833 23760 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:57:47.592744 23778 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:47.594267 23760 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:47.594455 23760 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c2e5ce4111f54d199ab7cbefde201f62"
format_stamp: "Formatted at 2026-08-09 07:57:43 on dist-test-slave-bw88"
I20260809 07:57:47.595136 23760 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:47.643306 23760 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:47.647267 23760 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:47.647580 23760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:47.648584 23760 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:47.650401 23760 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:47.650516 23760 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:47.650663 23760 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:47.650738 23760 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:57:47.651160 23760 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260809 07:57:47.651250 23760 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260809 07:57:47.651335 23760 fault_injection.cc:59] injecting fault for kudu (pid 23760): FLAGS_fault_before_start (process will exit)
I20260809 07:57:47.680884 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:46463
--local_ip_for_outbound_sockets=127.26.37.1
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991
--webserver_port=33301
--webserver_interface=127.26.37.1
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615 with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:48.051807 23789 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:48.052131 23789 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:48.052305 23789 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:48.060613 23789 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:48.060851 23789 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:48.071604 23789 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36365,127.26.37.21:33309,127.26.37.22:42445,127.26.37.23:34759,127.26.37.24:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:46463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=33301
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:43023,127.26.37.61:43433,127.26.37.60:39991,127.26.37.59:36615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:48.073307 23789 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:48.074779 23789 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:48.084985 23799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:48.085633 23800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:48.086019 23804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:48.086308 23789 server_base.cc:1061] running on GCE node
I20260809 07:57:48.086942 23789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:48.088114 23789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:48.089323 23789 hybrid_clock.cc:648] HybridClock initialized: now 1786262268089143 us; error 147 us; skew 500 ppm
I20260809 07:57:48.089789 23789 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:48 dist-test-slave-bw88 krb5kdc[21969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262268, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:48.093072 23789 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:48.094878 23789 webserver.cc:533] Webserver started at http://127.26.37.1:33301/ using document root <none> and password file <none>
I20260809 07:57:48.095631 23789 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:48.095781 23789 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:48.101784 23789 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 07:57:48.105150 23815 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:48.106642 23789 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:48.106788 23789 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "c2e5ce4111f54d199ab7cbefde201f62"
format_stamp: "Formatted at 2026-08-09 07:57:43 on dist-test-slave-bw88"
I20260809 07:57:48.107486 23789 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:48.128994 23789 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:48.144312 23789 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:48.144586 23789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:48.145658 23789 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:48.147317 23789 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:48.147404 23789 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:48.147483 23789 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:48.147559 23789 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:48.184265 23789 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:46463
I20260809 07:57:48.184321 23965 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:46463 every 8 connection(s)
I20260809 07:57:48.185879 23789 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:48.189008 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 23789
I20260809 07:57:48.189385 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23789
W20260809 07:57:48.204285 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23789: Not authorized: Client connection negotiation failed: client connection to 127.26.37.1:46463: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:57:48.204424 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 23789
I20260809 07:57:48.228034 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23419
W20260809 07:57:48.228804 23792 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:48.200959 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:56313 (local address 127.26.37.62:43023)
0809 07:57:48.201151 (+ 192us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:48.201170 (+ 19us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:48.202225 (+ 1055us) server_negotiation.cc:408] Connection header received
0809 07:57:48.202345 (+ 120us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:48.202351 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:48.202445 (+ 94us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:48.202558 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:48.227797 (+ 25239us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:56313: BlockingRecv error: recv got EOF from 127.26.37.1:56313 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 07:57:48.228888 23977 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:48.190791 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:44967 (local address 127.26.37.60:39991)
0809 07:57:48.195253 (+ 4462us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:48.195263 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:48.195289 (+ 26us) server_negotiation.cc:408] Connection header received
0809 07:57:48.195973 (+ 684us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:48.195983 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:48.196095 (+ 112us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:48.196236 (+ 141us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:48.199865 (+ 3629us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:57:48.201428 (+ 1563us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:57:48.227824 (+ 26396us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:44967: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":4327,"thread_start_us":4211,"threads_started":1}
W20260809 07:57:48.228938 23979 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:48.191610 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:43017 (local address 127.26.37.59:36615)
0809 07:57:48.196428 (+ 4818us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:48.196435 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:48.201745 (+ 5310us) server_negotiation.cc:408] Connection header received
0809 07:57:48.202054 (+ 309us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:48.202059 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:48.202129 (+ 70us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:48.202256 (+ 127us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:48.227839 (+ 25583us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:43017: BlockingRecv error: recv got EOF from 127.26.37.1:43017 (error 108)
Metrics: {"server-negotiator.queue_time_us":4676,"thread_start_us":4533,"threads_started":1}
W20260809 07:57:48.230599 23975 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:48.190475 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.26.37.1:34537 (local address 127.26.37.61:43433)
0809 07:57:48.191366 (+ 891us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:48.191376 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:48.200683 (+ 9307us) server_negotiation.cc:408] Connection header received
0809 07:57:48.200844 (+ 161us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:48.200852 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:48.200935 (+ 83us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:48.201071 (+ 136us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:48.203298 (+ 2227us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0809 07:57:48.204946 (+ 1648us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:57:48.229703 (+ 24757us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.37.1:34537: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":551,"threads_started":1}
W20260809 07:57:48.230949 23792 negotiation.cc:336] Failed RPC negotiation. Trace:
0809 07:57:48.229305 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49616 (local address 127.26.37.62:43023)
0809 07:57:48.229549 (+ 244us) server_negotiation.cc:207] Beginning negotiation
0809 07:57:48.229557 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0809 07:57:48.229583 (+ 26us) server_negotiation.cc:408] Connection header received
0809 07:57:48.229755 (+ 172us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0809 07:57:48.229760 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0809 07:57:48.229841 (+ 81us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0809 07:57:48.229934 (+ 93us) server_negotiation.cc:227] Negotiated authn=SASL
0809 07:57:48.230786 (+ 852us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49616: BlockingRecv error: recv got EOF from 127.0.0.1:49616 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
W20260809 07:57:48.230967 26772 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 23419: Not authorized: Client connection negotiation failed: client connection to 127.26.37.62:43023: server requires authentication, but client does not have Kerberos credentials available
I20260809 07:57:48.231104 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 23419
I20260809 07:57:48.266943 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22506
I20260809 07:57:48.268352 22588 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 07:57:48.340270 22641 consensus_peers.cc:597] T 00000000000000000000000000000000 P 981fdec031e74ccf93b919d9882446a7 -> Peer 0b91d76e03c64b0c80139683c4ff9283 (127.26.37.62:43023): Couldn't send request to peer 0b91d76e03c64b0c80139683c4ff9283. Status: Network error: Client connection negotiation failed: client connection to 127.26.37.62:43023: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:57:48.403795 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22506
I20260809 07:57:48.434914 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22612
I20260809 07:57:48.436118 22701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:48.552310 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22612
W20260809 07:57:48.581590 23145 connection.cc:573] server connection from 127.0.0.1:35162 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:57:48.582015 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 22986
I20260809 07:57:48.583273 23205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:48.696394 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 22986
Shutting down hive metastore.
2026-08-09T07:57:49Z chronyd exiting
2026-08-09T07:57:49Z chronyd exiting
2026-08-09T07:57:49Z chronyd exiting
2026-08-09T07:57:49Z chronyd exiting
2026-08-09T07:57:49Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/11 (27429 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/0
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:57:49 dist-test-slave-bw88 krb5kdc[24000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:57:49 dist-test-slave-bw88 krb5kdc[24000](info): set up 2 sockets
Aug 09 07:57:49 dist-test-slave-bw88 krb5kdc[24000](info): commencing operation
krb5kdc: starting...
W20260809 07:57:51.412010 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.221s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:57:51 dist-test-slave-bw88 krb5kdc[24000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:57:51Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:57:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:51.825019 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40553
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:40553
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:52.185158 24019 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:52.185563 24019 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:52.185730 24019 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:52.193744 24019 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:52.193832 24019 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:52.193971 24019 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:52.194054 24019 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:52.204685 24019 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:40553
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:40553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:52.206256 24019 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:52.207682 24019 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:52.216028 24025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:52.215991 24028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:52.216722 24030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:52.217329 24019 server_base.cc:1061] running on GCE node
I20260809 07:57:52.218083 24019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:52.219449 24019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:52.220688 24019 hybrid_clock.cc:648] HybridClock initialized: now 1786262272220649 us; error 50 us; skew 500 ppm
I20260809 07:57:52.221045 24019 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:52 dist-test-slave-bw88 krb5kdc[24000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262272, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:52.224184 24019 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:57:52.225802 24019 webserver.cc:533] Webserver started at http://127.26.37.62:32897/ using document root <none> and password file <none>
I20260809 07:57:52.226502 24019 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:52.226634 24019 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:52.226980 24019 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:52.229221 24019 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd"
format_stamp: "Formatted at 2026-08-09 07:57:52 on dist-test-slave-bw88"
I20260809 07:57:52.230039 24019 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd"
format_stamp: "Formatted at 2026-08-09 07:57:52 on dist-test-slave-bw88"
I20260809 07:57:52.235116 24019 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:57:52.238518 24043 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:52.239774 24019 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:57:52.239959 24019 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd"
format_stamp: "Formatted at 2026-08-09 07:57:52 on dist-test-slave-bw88"
I20260809 07:57:52.240156 24019 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:52.260417 24019 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:52.263999 24019 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:52.264295 24019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:52.285218 24019 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:40553
I20260809 07:57:52.285252 24124 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:40553 every 8 connection(s)
I20260809 07:57:52.286744 24019 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:57:52.290431 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24019
I20260809 07:57:52.290671 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:57:52.291201 24125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:52.303891 24125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd: Bootstrap starting.
Aug 09 07:57:52 dist-test-slave-bw88 krb5kdc[24000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:52.307507 24125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:52.308574 24125 log.cc:826] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:52.311702 24125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd: No bootstrap required, opened a new log
I20260809 07:57:52.317682 24125 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40553 } }
I20260809 07:57:52.318053 24125 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:52.318166 24125 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e55dd35a49f40ba8f3e5b542ac60dcd, State: Initialized, Role: FOLLOWER
I20260809 07:57:52.318817 24125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40553 } }
I20260809 07:57:52.319051 24125 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:57:52.319187 24125 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:57:52.319358 24125 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:57:52.321554 24125 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40553 } }
I20260809 07:57:52.322038 24125 leader_election.cc:304] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e55dd35a49f40ba8f3e5b542ac60dcd; no voters:
I20260809 07:57:52.322654 24125 leader_election.cc:290] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:57:52.322829 24131 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:57:52.324277 24131 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [term 1 LEADER]: Becoming Leader. State: Replica: 0e55dd35a49f40ba8f3e5b542ac60dcd, State: Running, Role: LEADER
I20260809 07:57:52.324757 24125 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:57:52.324739 24131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40553 } }
I20260809 07:57:52.329052 24135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e55dd35a49f40ba8f3e5b542ac60dcd. Latest consensus state: current_term: 1 leader_uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40553 } } }
I20260809 07:57:52.329458 24135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:52.329651 24132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e55dd35a49f40ba8f3e5b542ac60dcd" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 40553 } } }
I20260809 07:57:52.329984 24132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:52.331378 24145 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:57:52.336973 24145 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:57:52.346498 24145 catalog_manager.cc:1383] Generated new cluster ID: e520bba9bedf465cad16b8f9899dd768
I20260809 07:57:52.346630 24145 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:57:52.362085 24145 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:57:52.363335 24145 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:57:52.370452 24145 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 0e55dd35a49f40ba8f3e5b542ac60dcd: Generated new TSK 0
I20260809 07:57:52.371294 24145 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:52.526954 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:40553
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:52.881774 24164 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:52.882267 24164 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:52.882448 24164 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:52.890884 24164 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:52.891114 24164 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:52.901422 24164 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:40553
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:52.902987 24164 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:52.904300 24164 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:52.913806 24169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:52.913903 24170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:52.914148 24172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:52.915212 24164 server_base.cc:1061] running on GCE node
I20260809 07:57:52.915759 24164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:52.916692 24164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:52.917917 24164 hybrid_clock.cc:648] HybridClock initialized: now 1786262272917880 us; error 58 us; skew 500 ppm
I20260809 07:57:52.918262 24164 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:52 dist-test-slave-bw88 krb5kdc[24000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262272, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:52.921447 24164 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:52.922995 24164 webserver.cc:533] Webserver started at http://127.26.37.1:38705/ using document root <none> and password file <none>
I20260809 07:57:52.923714 24164 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:52.923851 24164 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:52.924182 24164 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:52.926649 24164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "2d379f1790614bd49bc892f6ca2f2643"
format_stamp: "Formatted at 2026-08-09 07:57:52 on dist-test-slave-bw88"
I20260809 07:57:52.927385 24164 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "2d379f1790614bd49bc892f6ca2f2643"
format_stamp: "Formatted at 2026-08-09 07:57:52 on dist-test-slave-bw88"
I20260809 07:57:52.932459 24164 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 07:57:52.935784 24183 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:52.937050 24164 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:57:52.937242 24164 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "2d379f1790614bd49bc892f6ca2f2643"
format_stamp: "Formatted at 2026-08-09 07:57:52 on dist-test-slave-bw88"
I20260809 07:57:52.937501 24164 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:52.954473 24164 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:52.958411 24164 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:52.958714 24164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:52.959678 24164 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:52.961230 24164 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:52.961334 24164 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:52.961517 24164 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:52.961603 24164 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:52.996668 24164 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43569
I20260809 07:57:52.996755 24350 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43569 every 8 connection(s)
I20260809 07:57:52.998191 24164 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:53.002178 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24164
I20260809 07:57:53.002435 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:57:53 dist-test-slave-bw88 krb5kdc[24000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262272, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:53.019107 24353 heartbeater.cc:344] Connected to a master server at 127.26.37.62:40553
I20260809 07:57:53.019529 24353 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:53.020377 24353 heartbeater.cc:507] Master 127.26.37.62:40553 requested a full tablet report, sending...
I20260809 07:57:53.022606 24068 ts_manager.cc:194] Registered new tserver with Master: 2d379f1790614bd49bc892f6ca2f2643 (127.26.37.1:43569)
I20260809 07:57:53.024097 24068 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:48121
I20260809 07:57:53.031025 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:57:53 dist-test-slave-bw88 krb5kdc[24000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:57:53 dist-test-slave-bw88 krb5kdc[24000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:57:53.108417 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24164
Aug 09 07:57:53 dist-test-slave-bw88 krb5kdc[24000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:57:53.125072 24346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:53.211064 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24164
I20260809 07:57:53.236179 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24019
I20260809 07:57:53.237648 24120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:53.345063 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24019
2026-08-09T07:57:53Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/0 (4206 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/1
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:57:53 dist-test-slave-bw88 krb5kdc[24369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:57:53 dist-test-slave-bw88 krb5kdc[24369](info): set up 2 sockets
Aug 09 07:57:53 dist-test-slave-bw88 krb5kdc[24369](info): commencing operation
krb5kdc: starting...
W20260809 07:57:55.584190 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.188s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:57:55 dist-test-slave-bw88 krb5kdc[24369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:57:55Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:57:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:56.020797 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:43105
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:43105
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:56.391841 24390 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:56.392198 24390 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:56.392283 24390 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:56.400825 24390 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:57:56.400933 24390 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:56.400997 24390 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:57:56.401046 24390 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:57:56.412293 24390 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:43105
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:43105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:56.413969 24390 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:56.415418 24390 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:56.424327 24396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:56.424628 24395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:56.425554 24399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:56.425935 24390 server_base.cc:1061] running on GCE node
I20260809 07:57:56.426784 24390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:56.428268 24390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:56.429497 24390 hybrid_clock.cc:648] HybridClock initialized: now 1786262276429450 us; error 55 us; skew 500 ppm
I20260809 07:57:56.429864 24390 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:56 dist-test-slave-bw88 krb5kdc[24369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262276, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:56.433128 24390 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:57:56.434985 24390 webserver.cc:533] Webserver started at http://127.26.37.62:43241/ using document root <none> and password file <none>
I20260809 07:57:56.435799 24390 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:56.435976 24390 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:56.436373 24390 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:56.439033 24390 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "ef80d00f12304c14b1f1b0458d98fba8"
format_stamp: "Formatted at 2026-08-09 07:57:56 on dist-test-slave-bw88"
I20260809 07:57:56.439908 24390 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "ef80d00f12304c14b1f1b0458d98fba8"
format_stamp: "Formatted at 2026-08-09 07:57:56 on dist-test-slave-bw88"
I20260809 07:57:56.445633 24390 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:57:56.449174 24410 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:56.450701 24390 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:57:56.450901 24390 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "ef80d00f12304c14b1f1b0458d98fba8"
format_stamp: "Formatted at 2026-08-09 07:57:56 on dist-test-slave-bw88"
I20260809 07:57:56.451128 24390 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:56.468359 24390 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:56.472404 24390 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:56.472712 24390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:56.495069 24390 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:43105
I20260809 07:57:56.495116 24482 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:43105 every 8 connection(s)
I20260809 07:57:56.496711 24390 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:57:56.498155 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24390
I20260809 07:57:56.498347 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:57:56.502578 24485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:57:56.512583 24485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8: Bootstrap starting.
Aug 09 07:57:56 dist-test-slave-bw88 krb5kdc[24369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:56.516327 24485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8: Neither blocks nor log segments found. Creating new log.
I20260809 07:57:56.517576 24485 log.cc:826] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8: Log is configured to *not* fsync() on all Append() calls
I20260809 07:57:56.520413 24485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8: No bootstrap required, opened a new log
I20260809 07:57:56.526650 24485 raft_consensus.cc:359] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef80d00f12304c14b1f1b0458d98fba8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43105 } }
I20260809 07:57:56.527078 24485 raft_consensus.cc:385] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:57:56.527212 24485 raft_consensus.cc:740] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef80d00f12304c14b1f1b0458d98fba8, State: Initialized, Role: FOLLOWER
I20260809 07:57:56.527810 24485 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef80d00f12304c14b1f1b0458d98fba8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43105 } }
I20260809 07:57:56.528034 24485 raft_consensus.cc:399] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:57:56.528174 24485 raft_consensus.cc:493] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:57:56.528352 24485 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:57:56.530778 24485 raft_consensus.cc:515] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef80d00f12304c14b1f1b0458d98fba8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43105 } }
I20260809 07:57:56.531311 24485 leader_election.cc:304] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef80d00f12304c14b1f1b0458d98fba8; no voters:
I20260809 07:57:56.531937 24485 leader_election.cc:290] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:57:56.532096 24490 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:57:56.532814 24490 raft_consensus.cc:697] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [term 1 LEADER]: Becoming Leader. State: Replica: ef80d00f12304c14b1f1b0458d98fba8, State: Running, Role: LEADER
I20260809 07:57:56.533443 24490 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef80d00f12304c14b1f1b0458d98fba8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43105 } }
I20260809 07:57:56.534192 24485 sys_catalog.cc:565] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:57:56.536615 24492 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef80d00f12304c14b1f1b0458d98fba8. Latest consensus state: current_term: 1 leader_uuid: "ef80d00f12304c14b1f1b0458d98fba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef80d00f12304c14b1f1b0458d98fba8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43105 } } }
I20260809 07:57:56.536989 24492 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:56.536801 24491 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef80d00f12304c14b1f1b0458d98fba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef80d00f12304c14b1f1b0458d98fba8" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43105 } } }
I20260809 07:57:56.537122 24491 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8 [sys.catalog]: This master's current role is: LEADER
I20260809 07:57:56.539433 24498 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:57:56.545197 24498 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:57:56.553785 24498 catalog_manager.cc:1383] Generated new cluster ID: 31f2e8bab93f423bac4abe3ab28d12f9
I20260809 07:57:56.553908 24498 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:57:56.580353 24498 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:57:56.582237 24498 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:57:56.594588 24498 catalog_manager.cc:6092] T 00000000000000000000000000000000 P ef80d00f12304c14b1f1b0458d98fba8: Generated new TSK 0
I20260809 07:57:56.595715 24498 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:57:56.779230 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:43105
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:57:57.148867 24517 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:57:57.149174 24517 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:57:57.149295 24517 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:57:57.157697 24517 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:57:57.157876 24517 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:57:57.168174 24517 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:43105
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:57:57.169791 24517 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:57:57.171479 24517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:57:57.181061 24528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:57.181056 24525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:57:57.181556 24524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:57:57.182315 24517 server_base.cc:1061] running on GCE node
I20260809 07:57:57.182889 24517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:57:57.183892 24517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:57:57.185128 24517 hybrid_clock.cc:648] HybridClock initialized: now 1786262277185086 us; error 55 us; skew 500 ppm
I20260809 07:57:57.185535 24517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:57:57 dist-test-slave-bw88 krb5kdc[24369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262277, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:57:57.188711 24517 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:57:57.190378 24517 webserver.cc:533] Webserver started at http://127.26.37.1:35775/ using document root <none> and password file <none>
I20260809 07:57:57.191138 24517 fs_manager.cc:362] Metadata directory not provided
I20260809 07:57:57.191287 24517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:57:57.191648 24517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:57:57.194125 24517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "e13ae134aef542dc835fa1f3e9e7c625"
format_stamp: "Formatted at 2026-08-09 07:57:57 on dist-test-slave-bw88"
I20260809 07:57:57.194916 24517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "e13ae134aef542dc835fa1f3e9e7c625"
format_stamp: "Formatted at 2026-08-09 07:57:57 on dist-test-slave-bw88"
I20260809 07:57:57.200405 24517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:57:57.203827 24538 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:57.205169 24517 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:57:57.205392 24517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "e13ae134aef542dc835fa1f3e9e7c625"
format_stamp: "Formatted at 2026-08-09 07:57:57 on dist-test-slave-bw88"
I20260809 07:57:57.205619 24517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:57:57.234297 24517 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:57:57.238427 24517 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:57:57.238751 24517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:57:57.239853 24517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:57:57.241608 24517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:57:57.241711 24517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:57.241859 24517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:57:57.241932 24517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:57:57.278448 24517 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:41715
I20260809 07:57:57.278544 24696 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:41715 every 8 connection(s)
I20260809 07:57:57.280006 24517 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:57:57.287314 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24517
I20260809 07:57:57.287537 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:57:57 dist-test-slave-bw88 krb5kdc[24369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262277, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:57:57.300859 24697 heartbeater.cc:344] Connected to a master server at 127.26.37.62:43105
I20260809 07:57:57.301286 24697 heartbeater.cc:461] Registering TS with master...
I20260809 07:57:57.302268 24697 heartbeater.cc:507] Master 127.26.37.62:43105 requested a full tablet report, sending...
I20260809 07:57:57.304639 24442 ts_manager.cc:194] Registered new tserver with Master: e13ae134aef542dc835fa1f3e9e7c625 (127.26.37.1:41715)
I20260809 07:57:57.306226 24442 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:35933
I20260809 07:57:57.315665 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:57:57 dist-test-slave-bw88 krb5kdc[24369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:57:57 dist-test-slave-bw88 krb5kdc[24369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:57:57.387521 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24517
Aug 09 07:57:57 dist-test-slave-bw88 krb5kdc[24369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:57:57.405411 24692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:57.491565 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24517
I20260809 07:57:57.514712 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24390
I20260809 07:57:57.515936 24478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:57:57.622610 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24390
2026-08-09T07:57:57Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/1 (4277 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/2
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:57:57 dist-test-slave-bw88 krb5kdc[24713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:57:57 dist-test-slave-bw88 krb5kdc[24713](info): set up 2 sockets
Aug 09 07:57:57 dist-test-slave-bw88 krb5kdc[24713](info): commencing operation
krb5kdc: starting...
W20260809 07:57:59.908038 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.235s user 0.005s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:00 dist-test-slave-bw88 krb5kdc[24713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:00.346645 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:43881
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:43881
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:00.712545 24736 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:00.712965 24736 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:00.713129 24736 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:00.721611 24736 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:00.721767 24736 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:00.721897 24736 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:00.721987 24736 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:00.732620 24736 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:43881
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:43881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:00.734383 24736 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:00.735836 24736 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:00.744078 24744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:00.744309 24745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:00.744951 24748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:00.745651 24736 server_base.cc:1061] running on GCE node
I20260809 07:58:00.746385 24736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:00.747743 24736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:00.748989 24736 hybrid_clock.cc:648] HybridClock initialized: now 1786262280748939 us; error 59 us; skew 500 ppm
I20260809 07:58:00.749372 24736 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:00 dist-test-slave-bw88 krb5kdc[24713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262280, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:00.752549 24736 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:00.754242 24736 webserver.cc:533] Webserver started at http://127.26.37.62:41167/ using document root <none> and password file <none>
I20260809 07:58:00.754961 24736 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:00.755064 24736 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:00.755378 24736 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:00.757773 24736 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "62705c6434c34443a140cedb0dd351ca"
format_stamp: "Formatted at 2026-08-09 07:58:00 on dist-test-slave-bw88"
I20260809 07:58:00.758582 24736 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "62705c6434c34443a140cedb0dd351ca"
format_stamp: "Formatted at 2026-08-09 07:58:00 on dist-test-slave-bw88"
I20260809 07:58:00.763906 24736 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:58:00.767251 24758 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:00.768550 24736 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:00.768738 24736 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "62705c6434c34443a140cedb0dd351ca"
format_stamp: "Formatted at 2026-08-09 07:58:00 on dist-test-slave-bw88"
I20260809 07:58:00.768898 24736 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:00.782872 24736 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:00.789826 24736 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:00.790134 24736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:00.811904 24736 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:43881
I20260809 07:58:00.811960 24825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:43881 every 8 connection(s)
I20260809 07:58:00.813537 24736 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:00.814211 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24736
I20260809 07:58:00.814486 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:00.819446 24829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 07:58:00 dist-test-slave-bw88 krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:00.831563 24829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca: Bootstrap starting.
I20260809 07:58:00.836053 24829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:00.837560 24829 log.cc:826] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:00.840503 24829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca: No bootstrap required, opened a new log
I20260809 07:58:00.845820 24829 raft_consensus.cc:359] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62705c6434c34443a140cedb0dd351ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43881 } }
I20260809 07:58:00.846185 24829 raft_consensus.cc:385] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:00.846328 24829 raft_consensus.cc:740] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62705c6434c34443a140cedb0dd351ca, State: Initialized, Role: FOLLOWER
I20260809 07:58:00.846966 24829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62705c6434c34443a140cedb0dd351ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43881 } }
I20260809 07:58:00.847204 24829 raft_consensus.cc:399] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:00.847349 24829 raft_consensus.cc:493] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:00.847517 24829 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:00.849756 24829 raft_consensus.cc:515] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62705c6434c34443a140cedb0dd351ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43881 } }
I20260809 07:58:00.850243 24829 leader_election.cc:304] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62705c6434c34443a140cedb0dd351ca; no voters:
I20260809 07:58:00.851457 24829 leader_election.cc:290] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:00.851572 24836 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:00.852303 24836 raft_consensus.cc:697] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [term 1 LEADER]: Becoming Leader. State: Replica: 62705c6434c34443a140cedb0dd351ca, State: Running, Role: LEADER
I20260809 07:58:00.852953 24836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62705c6434c34443a140cedb0dd351ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43881 } }
I20260809 07:58:00.853977 24829 sys_catalog.cc:565] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:00.856180 24838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62705c6434c34443a140cedb0dd351ca. Latest consensus state: current_term: 1 leader_uuid: "62705c6434c34443a140cedb0dd351ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62705c6434c34443a140cedb0dd351ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43881 } } }
I20260809 07:58:00.856496 24838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:00.856963 24837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62705c6434c34443a140cedb0dd351ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62705c6434c34443a140cedb0dd351ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 43881 } } }
I20260809 07:58:00.857270 24837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:00.857648 24845 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:00.864604 24845 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:00.873730 24845 catalog_manager.cc:1383] Generated new cluster ID: a1268662ff554e6cbf04334acb0bb7db
I20260809 07:58:00.873943 24845 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:00.926331 24845 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:00.928181 24845 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:00.941460 24845 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 62705c6434c34443a140cedb0dd351ca: Generated new TSK 0
I20260809 07:58:00.942569 24845 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:01.098546 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:43881
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:01.461681 24863 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:01.462002 24863 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:01.462095 24863 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:01.470177 24863 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:01.470504 24863 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:01.480863 24863 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:43881
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:01.482640 24863 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:01.484087 24863 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:01.494055 24875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:01.494020 24871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:01.494620 24870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:01.495250 24863 server_base.cc:1061] running on GCE node
I20260809 07:58:01.495847 24863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:01.496810 24863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:01.498101 24863 hybrid_clock.cc:648] HybridClock initialized: now 1786262281498065 us; error 57 us; skew 500 ppm
I20260809 07:58:01.498486 24863 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:01 dist-test-slave-bw88 krb5kdc[24713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262281, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:01.501914 24863 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:01.503621 24863 webserver.cc:533] Webserver started at http://127.26.37.1:32999/ using document root <none> and password file <none>
I20260809 07:58:01.504424 24863 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:01.504575 24863 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:01.504952 24863 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:01.507608 24863 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "4dbe0b8ce11341b99b9b9adfd278afdc"
format_stamp: "Formatted at 2026-08-09 07:58:01 on dist-test-slave-bw88"
I20260809 07:58:01.508677 24863 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "4dbe0b8ce11341b99b9b9adfd278afdc"
format_stamp: "Formatted at 2026-08-09 07:58:01 on dist-test-slave-bw88"
I20260809 07:58:01.514819 24863 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 07:58:01.518817 24883 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:01.520365 24863 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:58:01.520637 24863 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "4dbe0b8ce11341b99b9b9adfd278afdc"
format_stamp: "Formatted at 2026-08-09 07:58:01 on dist-test-slave-bw88"
I20260809 07:58:01.520897 24863 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:01.536666 24863 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:01.541002 24863 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:01.541467 24863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:01.542578 24863 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:01.544513 24863 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:01.544629 24863 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:01.544803 24863 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:01.544929 24863 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 07:58:01.583191 24863 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34283
I20260809 07:58:01.583240 25046 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34283 every 8 connection(s)
I20260809 07:58:01.584707 24863 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:01.586424 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 24863
I20260809 07:58:01.586637 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:01 dist-test-slave-bw88 krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262281, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:01.606231 25048 heartbeater.cc:344] Connected to a master server at 127.26.37.62:43881
I20260809 07:58:01.606554 25048 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:01.607342 25048 heartbeater.cc:507] Master 127.26.37.62:43881 requested a full tablet report, sending...
I20260809 07:58:01.609524 24779 ts_manager.cc:194] Registered new tserver with Master: 4dbe0b8ce11341b99b9b9adfd278afdc (127.26.37.1:34283)
I20260809 07:58:01.610997 24779 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:32929
I20260809 07:58:01.615468 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:01 dist-test-slave-bw88 krb5kdc[24713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:01 dist-test-slave-bw88 krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:01.687747 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24863
Aug 09 07:58:01 dist-test-slave-bw88 krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:01.705636 25037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:01.790503 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24863
I20260809 07:58:01.813962 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 24736
I20260809 07:58:01.815236 24818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:01.924077 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 24736
2026-08-09T07:58:01Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/2 (4301 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/3
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:02 dist-test-slave-bw88 krb5kdc[25063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:02 dist-test-slave-bw88 krb5kdc[25063](info): set up 2 sockets
Aug 09 07:58:02 dist-test-slave-bw88 krb5kdc[25063](info): commencing operation
krb5kdc: starting...
W20260809 07:58:04.254522 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.279s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:04 dist-test-slave-bw88 krb5kdc[25063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:04Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:04.700537 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:44917
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:44917
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:05.057497 25085 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:05.057824 25085 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:05.057993 25085 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:05.066293 25085 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:05.066378 25085 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:05.066494 25085 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:05.066563 25085 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:05.076862 25085 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:44917
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:44917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:05.078454 25085 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:05.079830 25085 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:05.088739 25094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:05.089468 25092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:05.089465 25091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:05.090373 25085 server_base.cc:1061] running on GCE node
I20260809 07:58:05.091306 25085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:05.093218 25085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:05.094473 25085 hybrid_clock.cc:648] HybridClock initialized: now 1786262285094343 us; error 99 us; skew 500 ppm
I20260809 07:58:05.094872 25085 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:05 dist-test-slave-bw88 krb5kdc[25063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262285, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:05.098104 25085 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:05.099762 25085 webserver.cc:533] Webserver started at http://127.26.37.62:41949/ using document root <none> and password file <none>
I20260809 07:58:05.100539 25085 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:05.100701 25085 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:05.101068 25085 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:05.103571 25085 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "48c28a845354441380efeb6e8b44d3cf"
format_stamp: "Formatted at 2026-08-09 07:58:05 on dist-test-slave-bw88"
I20260809 07:58:05.104470 25085 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "48c28a845354441380efeb6e8b44d3cf"
format_stamp: "Formatted at 2026-08-09 07:58:05 on dist-test-slave-bw88"
I20260809 07:58:05.110335 25085 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 07:58:05.113649 25105 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:05.114971 25085 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:58:05.115162 25085 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "48c28a845354441380efeb6e8b44d3cf"
format_stamp: "Formatted at 2026-08-09 07:58:05 on dist-test-slave-bw88"
I20260809 07:58:05.115347 25085 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:05.150708 25085 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:05.155257 25085 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:05.155609 25085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:05.178543 25085 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:44917
I20260809 07:58:05.178587 25175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:44917 every 8 connection(s)
I20260809 07:58:05.180290 25085 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:05.184841 25176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:05.185214 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25085
I20260809 07:58:05.185441 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:05.197991 25176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf: Bootstrap starting.
Aug 09 07:58:05 dist-test-slave-bw88 krb5kdc[25063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:05.201728 25176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:05.202811 25176 log.cc:826] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:05.206073 25176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf: No bootstrap required, opened a new log
I20260809 07:58:05.212040 25176 raft_consensus.cc:359] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c28a845354441380efeb6e8b44d3cf" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44917 } }
I20260809 07:58:05.212404 25176 raft_consensus.cc:385] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:05.212518 25176 raft_consensus.cc:740] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c28a845354441380efeb6e8b44d3cf, State: Initialized, Role: FOLLOWER
I20260809 07:58:05.213092 25176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c28a845354441380efeb6e8b44d3cf" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44917 } }
I20260809 07:58:05.213317 25176 raft_consensus.cc:399] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:05.213486 25176 raft_consensus.cc:493] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:05.213660 25176 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:05.215714 25176 raft_consensus.cc:515] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c28a845354441380efeb6e8b44d3cf" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44917 } }
I20260809 07:58:05.216200 25176 leader_election.cc:304] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48c28a845354441380efeb6e8b44d3cf; no voters:
I20260809 07:58:05.216763 25176 leader_election.cc:290] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:05.216910 25183 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:05.217669 25183 raft_consensus.cc:697] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [term 1 LEADER]: Becoming Leader. State: Replica: 48c28a845354441380efeb6e8b44d3cf, State: Running, Role: LEADER
I20260809 07:58:05.218274 25183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c28a845354441380efeb6e8b44d3cf" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44917 } }
I20260809 07:58:05.218866 25176 sys_catalog.cc:565] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:05.221608 25186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48c28a845354441380efeb6e8b44d3cf. Latest consensus state: current_term: 1 leader_uuid: "48c28a845354441380efeb6e8b44d3cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c28a845354441380efeb6e8b44d3cf" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44917 } } }
I20260809 07:58:05.221642 25185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48c28a845354441380efeb6e8b44d3cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c28a845354441380efeb6e8b44d3cf" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44917 } } }
I20260809 07:58:05.222008 25185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:05.222008 25186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf [sys.catalog]: This master's current role is: LEADER
W20260809 07:58:05.233451 25206 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:58:05.233585 25206 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:58:05.233711 25191 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:05.235447 25191 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:05.244143 25191 catalog_manager.cc:1383] Generated new cluster ID: 74f05dd1b7ec45b2b9d27634895e8c53
I20260809 07:58:05.244256 25191 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:05.255417 25191 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:05.257025 25191 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:05.269317 25191 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 48c28a845354441380efeb6e8b44d3cf: Generated new TSK 0
I20260809 07:58:05.270331 25191 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:05.420982 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:44917
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:05.777735 25217 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:05.778048 25217 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:05.778203 25217 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:05.786363 25217 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:05.786621 25217 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:05.797328 25217 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:44917
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:05.798985 25217 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:05.800345 25217 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:05.809628 25226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:05.809706 25227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:05.810451 25229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:05.810840 25217 server_base.cc:1061] running on GCE node
I20260809 07:58:05.811419 25217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:05.812345 25217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:05.813570 25217 hybrid_clock.cc:648] HybridClock initialized: now 1786262285813530 us; error 51 us; skew 500 ppm
I20260809 07:58:05.813918 25217 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:05 dist-test-slave-bw88 krb5kdc[25063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262285, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:05.817014 25217 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:05.818545 25217 webserver.cc:533] Webserver started at http://127.26.37.1:45273/ using document root <none> and password file <none>
I20260809 07:58:05.819190 25217 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:05.819281 25217 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:05.819578 25217 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:05.822249 25217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "98979a0ddb214ceeb512b4dbc9aa58d3"
format_stamp: "Formatted at 2026-08-09 07:58:05 on dist-test-slave-bw88"
I20260809 07:58:05.823001 25217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "98979a0ddb214ceeb512b4dbc9aa58d3"
format_stamp: "Formatted at 2026-08-09 07:58:05 on dist-test-slave-bw88"
I20260809 07:58:05.828485 25217 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:58:05.831938 25238 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:05.833328 25217 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:58:05.833587 25217 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "98979a0ddb214ceeb512b4dbc9aa58d3"
format_stamp: "Formatted at 2026-08-09 07:58:05 on dist-test-slave-bw88"
I20260809 07:58:05.833789 25217 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:05.855381 25217 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:05.862849 25217 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:05.863178 25217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:05.864161 25217 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:05.865849 25217 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:05.865967 25217 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:05.866142 25217 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:05.866217 25217 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:05.901304 25217 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:41727
I20260809 07:58:05.901435 25386 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:41727 every 8 connection(s)
I20260809 07:58:05.902781 25217 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:05.905643 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25217
I20260809 07:58:05.905844 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:05 dist-test-slave-bw88 krb5kdc[25063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262285, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:05.926026 25387 heartbeater.cc:344] Connected to a master server at 127.26.37.62:44917
I20260809 07:58:05.926349 25387 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:05.927150 25387 heartbeater.cc:507] Master 127.26.37.62:44917 requested a full tablet report, sending...
I20260809 07:58:05.929260 25126 ts_manager.cc:194] Registered new tserver with Master: 98979a0ddb214ceeb512b4dbc9aa58d3 (127.26.37.1:41727)
I20260809 07:58:05.930871 25126 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:34759
I20260809 07:58:05.936766 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:05 dist-test-slave-bw88 krb5kdc[25063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:05 dist-test-slave-bw88 krb5kdc[25063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:06.006438 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25217
Aug 09 07:58:06 dist-test-slave-bw88 krb5kdc[25063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:06.023700 25379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:06.107515 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25217
I20260809 07:58:06.129238 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25085
I20260809 07:58:06.130571 25168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:06.234982 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25085
2026-08-09T07:58:06Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/3 (4306 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/4
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:06 dist-test-slave-bw88 krb5kdc[25406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:06 dist-test-slave-bw88 krb5kdc[25406](info): set up 2 sockets
Aug 09 07:58:06 dist-test-slave-bw88 krb5kdc[25406](info): commencing operation
krb5kdc: starting...
W20260809 07:58:08.490697 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.210s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:08 dist-test-slave-bw88 krb5kdc[25406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:08.935948 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:42917
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:45279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:42917
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:09.285604 25429 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:09.285897 25429 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:09.285974 25429 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:09.294090 25429 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:09.294186 25429 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:09.294250 25429 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:09.294291 25429 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:09.304580 25429 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:42917
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:09.306180 25429 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:09.307582 25429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:09.315878 25436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:09.315904 25437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:09.316380 25439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:09.317044 25429 server_base.cc:1061] running on GCE node
I20260809 07:58:09.317824 25429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:09.319223 25429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:09.320456 25429 hybrid_clock.cc:648] HybridClock initialized: now 1786262289320384 us; error 60 us; skew 500 ppm
I20260809 07:58:09.320794 25429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:09 dist-test-slave-bw88 krb5kdc[25406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262289, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:09.323922 25429 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:09.325660 25429 webserver.cc:533] Webserver started at http://127.26.37.62:34545/ using document root <none> and password file <none>
I20260809 07:58:09.326359 25429 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:09.326496 25429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:09.326839 25429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:09.329069 25429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "cb3cf4fae33146e5a9ec7727086f1e9b"
format_stamp: "Formatted at 2026-08-09 07:58:09 on dist-test-slave-bw88"
I20260809 07:58:09.329916 25429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "cb3cf4fae33146e5a9ec7727086f1e9b"
format_stamp: "Formatted at 2026-08-09 07:58:09 on dist-test-slave-bw88"
I20260809 07:58:09.334930 25429 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 07:58:09.338189 25446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:09.339471 25429 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:09.339645 25429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "cb3cf4fae33146e5a9ec7727086f1e9b"
format_stamp: "Formatted at 2026-08-09 07:58:09 on dist-test-slave-bw88"
I20260809 07:58:09.339833 25429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:09.370230 25429 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:09.374024 25429 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:09.374310 25429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:09.398514 25429 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:42917
I20260809 07:58:09.398556 25511 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:42917 every 8 connection(s)
I20260809 07:58:09.400058 25429 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:09.404330 25512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:09.406703 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25429
I20260809 07:58:09.406888 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:09.414695 25512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b: Bootstrap starting.
I20260809 07:58:09.418134 25512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b: Neither blocks nor log segments found. Creating new log.
Aug 09 07:58:09 dist-test-slave-bw88 krb5kdc[25406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:09.419386 25512 log.cc:826] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:09.422756 25512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b: No bootstrap required, opened a new log
I20260809 07:58:09.429872 25512 raft_consensus.cc:359] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cf4fae33146e5a9ec7727086f1e9b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42917 } }
I20260809 07:58:09.430281 25512 raft_consensus.cc:385] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:09.430388 25512 raft_consensus.cc:740] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3cf4fae33146e5a9ec7727086f1e9b, State: Initialized, Role: FOLLOWER
I20260809 07:58:09.430928 25512 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cf4fae33146e5a9ec7727086f1e9b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42917 } }
I20260809 07:58:09.431149 25512 raft_consensus.cc:399] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:09.431277 25512 raft_consensus.cc:493] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:09.431442 25512 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:09.433755 25512 raft_consensus.cc:515] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cf4fae33146e5a9ec7727086f1e9b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42917 } }
I20260809 07:58:09.434252 25512 leader_election.cc:304] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb3cf4fae33146e5a9ec7727086f1e9b; no voters:
I20260809 07:58:09.434834 25512 leader_election.cc:290] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:09.434969 25519 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:09.435748 25519 raft_consensus.cc:697] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [term 1 LEADER]: Becoming Leader. State: Replica: cb3cf4fae33146e5a9ec7727086f1e9b, State: Running, Role: LEADER
I20260809 07:58:09.436231 25519 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cf4fae33146e5a9ec7727086f1e9b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42917 } }
I20260809 07:58:09.436978 25512 sys_catalog.cc:565] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:09.439291 25522 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb3cf4fae33146e5a9ec7727086f1e9b. Latest consensus state: current_term: 1 leader_uuid: "cb3cf4fae33146e5a9ec7727086f1e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cf4fae33146e5a9ec7727086f1e9b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42917 } } }
I20260809 07:58:09.439599 25522 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:09.441556 25520 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb3cf4fae33146e5a9ec7727086f1e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cf4fae33146e5a9ec7727086f1e9b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42917 } } }
I20260809 07:58:09.441900 25520 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:09.443403 25529 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:09.450065 25529 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:09.458630 25529 catalog_manager.cc:1383] Generated new cluster ID: 05649890e8ad4d72bec9911ab1f562d4
I20260809 07:58:09.458738 25529 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:09.470602 25529 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:09.472190 25529 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:09.483351 25529 catalog_manager.cc:6092] T 00000000000000000000000000000000 P cb3cf4fae33146e5a9ec7727086f1e9b: Generated new TSK 0
I20260809 07:58:09.484288 25529 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:09.636806 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:42917
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:45279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:09.992339 25551 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:09.992662 25551 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:09.992830 25551 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:10.001044 25551 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:10.001258 25551 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:10.011953 25551 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:45279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:42917
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:10.013741 25551 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:10.015169 25551 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:10.025605 25559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:10.025636 25557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:10.026767 25561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:10.026784 25551 server_base.cc:1061] running on GCE node
I20260809 07:58:10.027446 25551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:10.028328 25551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:10.029531 25551 hybrid_clock.cc:648] HybridClock initialized: now 1786262290029478 us; error 43 us; skew 500 ppm
I20260809 07:58:10.029814 25551 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:10 dist-test-slave-bw88 krb5kdc[25406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262290, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:10.033146 25551 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:10.034842 25551 webserver.cc:533] Webserver started at http://127.26.37.1:37381/ using document root <none> and password file <none>
I20260809 07:58:10.035557 25551 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:10.035653 25551 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:10.035940 25551 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:10.038561 25551 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "ab6c18d49adc41fab16b2a0622336603"
format_stamp: "Formatted at 2026-08-09 07:58:10 on dist-test-slave-bw88"
I20260809 07:58:10.039299 25551 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "ab6c18d49adc41fab16b2a0622336603"
format_stamp: "Formatted at 2026-08-09 07:58:10 on dist-test-slave-bw88"
I20260809 07:58:10.044986 25551 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 07:58:10.048564 25570 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:10.050028 25551 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:10.050212 25551 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "ab6c18d49adc41fab16b2a0622336603"
format_stamp: "Formatted at 2026-08-09 07:58:10 on dist-test-slave-bw88"
I20260809 07:58:10.050448 25551 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:10.064392 25551 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:10.068085 25551 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:10.068385 25551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:10.069303 25551 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:10.070870 25551 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:10.070981 25551 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:10.071131 25551 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:10.071206 25551 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:10.106957 25551 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43289
I20260809 07:58:10.107050 25726 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43289 every 8 connection(s)
I20260809 07:58:10.108487 25551 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:10.112567 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25551
I20260809 07:58:10.112783 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:10 dist-test-slave-bw88 krb5kdc[25406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262290, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:10.129905 25727 heartbeater.cc:344] Connected to a master server at 127.26.37.62:42917
I20260809 07:58:10.130347 25727 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:10.131496 25727 heartbeater.cc:507] Master 127.26.37.62:42917 requested a full tablet report, sending...
I20260809 07:58:10.134157 25465 ts_manager.cc:194] Registered new tserver with Master: ab6c18d49adc41fab16b2a0622336603 (127.26.37.1:43289)
I20260809 07:58:10.136016 25465 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:34725
I20260809 07:58:10.148033 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:10 dist-test-slave-bw88 krb5kdc[25406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:10 dist-test-slave-bw88 krb5kdc[25406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:10.242760 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25551
Aug 09 07:58:10 dist-test-slave-bw88 krb5kdc[25406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:10.262612 25720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:10.349287 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25551
I20260809 07:58:10.375633 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25429
I20260809 07:58:10.376840 25504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:10.484095 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25429
2026-08-09T07:58:10Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/4 (4250 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/5
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:10 dist-test-slave-bw88 krb5kdc[25742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:10 dist-test-slave-bw88 krb5kdc[25742](info): set up 2 sockets
Aug 09 07:58:10 dist-test-slave-bw88 krb5kdc[25742](info): commencing operation
krb5kdc: starting...
W20260809 07:58:12.737038 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.205s user 0.000s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:12 dist-test-slave-bw88 krb5kdc[25742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:12Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:13.166887 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36619
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:36619
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:13.522878 25771 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:13.523236 25771 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:13.523406 25771 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:13.532006 25771 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:13.532096 25771 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:13.532203 25771 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:13.532254 25771 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:13.542687 25771 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:36619
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:36619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:13.544368 25771 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:13.545885 25771 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:13.554351 25776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:13.554497 25777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:13.554913 25779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:13.556725 25771 server_base.cc:1061] running on GCE node
I20260809 07:58:13.557526 25771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:13.558900 25771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:13.560140 25771 hybrid_clock.cc:648] HybridClock initialized: now 1786262293560091 us; error 65 us; skew 500 ppm
I20260809 07:58:13.560483 25771 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:13 dist-test-slave-bw88 krb5kdc[25742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262293, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:13.563740 25771 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:13.565563 25771 webserver.cc:533] Webserver started at http://127.26.37.62:43895/ using document root <none> and password file <none>
I20260809 07:58:13.566303 25771 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:13.566452 25771 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:13.566846 25771 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:13.569425 25771 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "2f0762e1080048598040cf105a0ed1be"
format_stamp: "Formatted at 2026-08-09 07:58:13 on dist-test-slave-bw88"
I20260809 07:58:13.570241 25771 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "2f0762e1080048598040cf105a0ed1be"
format_stamp: "Formatted at 2026-08-09 07:58:13 on dist-test-slave-bw88"
I20260809 07:58:13.575747 25771 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 07:58:13.579295 25790 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:13.580571 25771 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:13.580770 25771 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "2f0762e1080048598040cf105a0ed1be"
format_stamp: "Formatted at 2026-08-09 07:58:13 on dist-test-slave-bw88"
I20260809 07:58:13.580961 25771 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:13.602066 25771 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:13.610803 25771 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:13.611109 25771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:13.632244 25771 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:36619
I20260809 07:58:13.632294 25860 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:36619 every 8 connection(s)
I20260809 07:58:13.633901 25771 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:13.638216 25861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:13.643647 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25771
I20260809 07:58:13.643868 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:13.649042 25861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be: Bootstrap starting.
I20260809 07:58:13.652325 25861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:13.653488 25861 log.cc:826] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:13.656836 25861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be: No bootstrap required, opened a new log
Aug 09 07:58:13 dist-test-slave-bw88 krb5kdc[25742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:13.662598 25861 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0762e1080048598040cf105a0ed1be" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36619 } }
I20260809 07:58:13.662973 25861 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:13.663141 25861 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f0762e1080048598040cf105a0ed1be, State: Initialized, Role: FOLLOWER
I20260809 07:58:13.663939 25861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0762e1080048598040cf105a0ed1be" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36619 } }
I20260809 07:58:13.664211 25861 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:13.664356 25861 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:13.664577 25861 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:13.666929 25861 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0762e1080048598040cf105a0ed1be" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36619 } }
I20260809 07:58:13.667416 25861 leader_election.cc:304] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f0762e1080048598040cf105a0ed1be; no voters:
I20260809 07:58:13.668042 25861 leader_election.cc:290] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:13.668176 25868 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:13.669438 25868 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [term 1 LEADER]: Becoming Leader. State: Replica: 2f0762e1080048598040cf105a0ed1be, State: Running, Role: LEADER
I20260809 07:58:13.670009 25868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0762e1080048598040cf105a0ed1be" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36619 } }
I20260809 07:58:13.670327 25861 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:13.672946 25871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f0762e1080048598040cf105a0ed1be. Latest consensus state: current_term: 1 leader_uuid: "2f0762e1080048598040cf105a0ed1be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0762e1080048598040cf105a0ed1be" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36619 } } }
I20260809 07:58:13.673266 25871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:13.673748 25869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f0762e1080048598040cf105a0ed1be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0762e1080048598040cf105a0ed1be" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 36619 } } }
I20260809 07:58:13.674079 25869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:13.676191 25877 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:13.681866 25877 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:13.690748 25877 catalog_manager.cc:1383] Generated new cluster ID: b11fe167348c473689d17dd708e3552e
I20260809 07:58:13.690858 25877 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:13.710070 25877 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:13.711269 25877 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:13.720314 25877 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 2f0762e1080048598040cf105a0ed1be: Generated new TSK 0
I20260809 07:58:13.721068 25877 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:13.882068 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:36619
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:14.237128 25900 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:14.237515 25900 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:14.237680 25900 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:14.245769 25900 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:14.245994 25900 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:14.256178 25900 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:36619
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:14.257790 25900 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:14.259104 25900 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:14.268168 25905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:14.268198 25906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:14.268918 25909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:14.269728 25900 server_base.cc:1061] running on GCE node
I20260809 07:58:14.270422 25900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:14.271557 25900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:14.272773 25900 hybrid_clock.cc:648] HybridClock initialized: now 1786262294272712 us; error 66 us; skew 500 ppm
I20260809 07:58:14.273104 25900 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:14 dist-test-slave-bw88 krb5kdc[25742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:14.276170 25900 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:14.277827 25900 webserver.cc:533] Webserver started at http://127.26.37.1:38081/ using document root <none> and password file <none>
I20260809 07:58:14.278559 25900 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:14.278695 25900 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:14.279048 25900 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:14.281424 25900 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "6ca950bc83e04e1b84eb14eb66380849"
format_stamp: "Formatted at 2026-08-09 07:58:14 on dist-test-slave-bw88"
I20260809 07:58:14.282214 25900 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "6ca950bc83e04e1b84eb14eb66380849"
format_stamp: "Formatted at 2026-08-09 07:58:14 on dist-test-slave-bw88"
I20260809 07:58:14.287523 25900 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:58:14.290813 25920 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:14.292135 25900 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:58:14.292318 25900 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "6ca950bc83e04e1b84eb14eb66380849"
format_stamp: "Formatted at 2026-08-09 07:58:14 on dist-test-slave-bw88"
I20260809 07:58:14.292529 25900 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:14.317648 25900 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:14.321182 25900 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:14.321534 25900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:14.322470 25900 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:14.323985 25900 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:14.324086 25900 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:14.324232 25900 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:14.324301 25900 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:14.359298 25900 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43657
I20260809 07:58:14.359355 26069 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43657 every 8 connection(s)
I20260809 07:58:14.360620 25900 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:14.365589 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 25900
I20260809 07:58:14.365767 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:14 dist-test-slave-bw88 krb5kdc[25742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:14.380511 26070 heartbeater.cc:344] Connected to a master server at 127.26.37.62:36619
I20260809 07:58:14.380931 26070 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:14.381853 26070 heartbeater.cc:507] Master 127.26.37.62:36619 requested a full tablet report, sending...
I20260809 07:58:14.383960 25814 ts_manager.cc:194] Registered new tserver with Master: 6ca950bc83e04e1b84eb14eb66380849 (127.26.37.1:43657)
I20260809 07:58:14.385576 25814 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:45851
I20260809 07:58:14.393405 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:14 dist-test-slave-bw88 krb5kdc[25742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:14 dist-test-slave-bw88 krb5kdc[25742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:14.461810 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25900
Aug 09 07:58:14 dist-test-slave-bw88 krb5kdc[25742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:14.478713 26064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:14.562747 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25900
I20260809 07:58:14.588574 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 25771
I20260809 07:58:14.589874 25854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:14.694567 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 25771
2026-08-09T07:58:14Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/5 (4212 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/6
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:14 dist-test-slave-bw88 krb5kdc[26086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:14 dist-test-slave-bw88 krb5kdc[26086](info): set up 2 sockets
Aug 09 07:58:14 dist-test-slave-bw88 krb5kdc[26086](info): commencing operation
krb5kdc: starting...
W20260809 07:58:16.958441 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.213s user 0.003s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:17 dist-test-slave-bw88 krb5kdc[26086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:17Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:17.366885 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:42413
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:42413
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:17.722307 26114 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:17.722692 26114 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:17.722882 26114 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:17.731267 26114 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:17.731396 26114 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:17.731539 26114 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:17.731621 26114 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:17.742118 26114 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:42413
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:42413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:17.743741 26114 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:17.745172 26114 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:17.753559 26120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:17.754339 26121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:17.757613 26114 server_base.cc:1061] running on GCE node
W20260809 07:58:17.758415 26123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:17.759238 26114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:17.760653 26114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:17.761904 26114 hybrid_clock.cc:648] HybridClock initialized: now 1786262297761864 us; error 46 us; skew 500 ppm
I20260809 07:58:17.762270 26114 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:17 dist-test-slave-bw88 krb5kdc[26086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262297, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:17.765610 26114 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:17.767270 26114 webserver.cc:533] Webserver started at http://127.26.37.62:45455/ using document root <none> and password file <none>
I20260809 07:58:17.767967 26114 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:17.768064 26114 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:17.768354 26114 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:17.770818 26114 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "02c49d80af0249288795fe1d3da9e404"
format_stamp: "Formatted at 2026-08-09 07:58:17 on dist-test-slave-bw88"
I20260809 07:58:17.771657 26114 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "02c49d80af0249288795fe1d3da9e404"
format_stamp: "Formatted at 2026-08-09 07:58:17 on dist-test-slave-bw88"
I20260809 07:58:17.777112 26114 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 07:58:17.780436 26133 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:17.781916 26114 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:17.782107 26114 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "02c49d80af0249288795fe1d3da9e404"
format_stamp: "Formatted at 2026-08-09 07:58:17 on dist-test-slave-bw88"
I20260809 07:58:17.782301 26114 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:17.802330 26114 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:17.806262 26114 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:17.806559 26114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:17.827376 26114 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:42413
I20260809 07:58:17.827423 26200 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:42413 every 8 connection(s)
I20260809 07:58:17.828917 26114 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:17.829711 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26114
I20260809 07:58:17.829936 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:17.834372 26201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 07:58:17 dist-test-slave-bw88 krb5kdc[26086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:17.843837 26201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404: Bootstrap starting.
I20260809 07:58:17.847407 26201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:17.848453 26201 log.cc:826] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:17.851886 26201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404: No bootstrap required, opened a new log
I20260809 07:58:17.857410 26201 raft_consensus.cc:359] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c49d80af0249288795fe1d3da9e404" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42413 } }
I20260809 07:58:17.857767 26201 raft_consensus.cc:385] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:17.857853 26201 raft_consensus.cc:740] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c49d80af0249288795fe1d3da9e404, State: Initialized, Role: FOLLOWER
I20260809 07:58:17.858420 26201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c49d80af0249288795fe1d3da9e404" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42413 } }
I20260809 07:58:17.858609 26201 raft_consensus.cc:399] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:17.858714 26201 raft_consensus.cc:493] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:17.858824 26201 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:17.861018 26201 raft_consensus.cc:515] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c49d80af0249288795fe1d3da9e404" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42413 } }
I20260809 07:58:17.861594 26201 leader_election.cc:304] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02c49d80af0249288795fe1d3da9e404; no voters:
I20260809 07:58:17.862154 26201 leader_election.cc:290] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:17.862308 26206 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:17.863029 26206 raft_consensus.cc:697] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [term 1 LEADER]: Becoming Leader. State: Replica: 02c49d80af0249288795fe1d3da9e404, State: Running, Role: LEADER
I20260809 07:58:17.863533 26206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c49d80af0249288795fe1d3da9e404" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42413 } }
I20260809 07:58:17.864400 26201 sys_catalog.cc:565] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:17.866904 26208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02c49d80af0249288795fe1d3da9e404. Latest consensus state: current_term: 1 leader_uuid: "02c49d80af0249288795fe1d3da9e404" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c49d80af0249288795fe1d3da9e404" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42413 } } }
I20260809 07:58:17.867237 26208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:17.869602 26207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02c49d80af0249288795fe1d3da9e404" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c49d80af0249288795fe1d3da9e404" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 42413 } } }
I20260809 07:58:17.870039 26207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404 [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:17.875283 26217 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:17.880746 26217 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:17.889299 26217 catalog_manager.cc:1383] Generated new cluster ID: 843bc5a48429459d92ea9e3df20a0a32
I20260809 07:58:17.889438 26217 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:17.909919 26217 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:17.911183 26217 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:17.923070 26217 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 02c49d80af0249288795fe1d3da9e404: Generated new TSK 0
I20260809 07:58:17.923899 26217 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:18.084321 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:42413
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:18.451859 26240 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:18.452154 26240 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:18.452351 26240 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:18.460942 26240 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:18.461185 26240 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:18.471825 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:42413
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:18.473526 26240 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:18.475008 26240 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:18.485270 26248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:18.485248 26246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:18.486085 26250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:18.486747 26240 server_base.cc:1061] running on GCE node
I20260809 07:58:18.487460 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:18.488447 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:18.489676 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786262298489615 us; error 56 us; skew 500 ppm
I20260809 07:58:18.490000 26240 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:18 dist-test-slave-bw88 krb5kdc[26086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262298, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:18.493119 26240 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:18.494699 26240 webserver.cc:533] Webserver started at http://127.26.37.1:42697/ using document root <none> and password file <none>
I20260809 07:58:18.495369 26240 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:18.495466 26240 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:18.495774 26240 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:18.498219 26240 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "8afeea60ff044865ac49a5417afdbdca"
format_stamp: "Formatted at 2026-08-09 07:58:18 on dist-test-slave-bw88"
I20260809 07:58:18.498950 26240 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "8afeea60ff044865ac49a5417afdbdca"
format_stamp: "Formatted at 2026-08-09 07:58:18 on dist-test-slave-bw88"
I20260809 07:58:18.504341 26240 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:58:18.507669 26261 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:18.508936 26240 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:58:18.509068 26240 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "8afeea60ff044865ac49a5417afdbdca"
format_stamp: "Formatted at 2026-08-09 07:58:18 on dist-test-slave-bw88"
I20260809 07:58:18.509225 26240 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:18.530599 26240 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:18.534199 26240 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:18.534431 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:18.535323 26240 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:18.536784 26240 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:18.536856 26240 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:18.536938 26240 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:18.536981 26240 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:18.574061 26240 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:33089
I20260809 07:58:18.574195 26419 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:33089 every 8 connection(s)
I20260809 07:58:18.576228 26240 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:18.581522 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26240
I20260809 07:58:18.581730 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:18 dist-test-slave-bw88 krb5kdc[26086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262298, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:18.603204 26420 heartbeater.cc:344] Connected to a master server at 127.26.37.62:42413
I20260809 07:58:18.603636 26420 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:18.604821 26420 heartbeater.cc:507] Master 127.26.37.62:42413 requested a full tablet report, sending...
I20260809 07:58:18.607374 26155 ts_manager.cc:194] Registered new tserver with Master: 8afeea60ff044865ac49a5417afdbdca (127.26.37.1:33089)
I20260809 07:58:18.609730 26155 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:57093
I20260809 07:58:18.612381 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:18 dist-test-slave-bw88 krb5kdc[26086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:18 dist-test-slave-bw88 krb5kdc[26086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:18.698633 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26240
Aug 09 07:58:18 dist-test-slave-bw88 krb5kdc[26086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:18.715194 26413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:18.801270 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26240
I20260809 07:58:18.826977 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26114
I20260809 07:58:18.828227 26195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:18.932565 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26114
2026-08-09T07:58:18Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/6 (4234 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/7
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:19 dist-test-slave-bw88 krb5kdc[26444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:19 dist-test-slave-bw88 krb5kdc[26444](info): set up 2 sockets
Aug 09 07:58:19 dist-test-slave-bw88 krb5kdc[26444](info): commencing operation
krb5kdc: starting...
W20260809 07:58:21.221045 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.242s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:21 dist-test-slave-bw88 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:21Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:21.628141 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34611
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:34611
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:21.975921 26470 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:21.976289 26470 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:21.976459 26470 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:21.984620 26470 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:21.984735 26470 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:21.984797 26470 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:21.984870 26470 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:21.995105 26470 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:34611
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:34611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:21.996698 26470 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:21.998138 26470 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:22.006563 26476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:22.006505 26475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:22.007332 26478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:22.007719 26470 server_base.cc:1061] running on GCE node
I20260809 07:58:22.008433 26470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:22.009806 26470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:22.011075 26470 hybrid_clock.cc:648] HybridClock initialized: now 1786262302011013 us; error 57 us; skew 500 ppm
I20260809 07:58:22.011413 26470 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:22 dist-test-slave-bw88 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262302, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:22.014614 26470 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:22.016338 26470 webserver.cc:533] Webserver started at http://127.26.37.62:34035/ using document root <none> and password file <none>
I20260809 07:58:22.017287 26470 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:22.017444 26470 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:22.017796 26470 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:22.020282 26470 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "6f2648e70c16418b949bde8d7ffffd2a"
format_stamp: "Formatted at 2026-08-09 07:58:22 on dist-test-slave-bw88"
I20260809 07:58:22.021103 26470 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "6f2648e70c16418b949bde8d7ffffd2a"
format_stamp: "Formatted at 2026-08-09 07:58:22 on dist-test-slave-bw88"
I20260809 07:58:22.026438 26470 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 07:58:22.029699 26491 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:22.030961 26470 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:58:22.031159 26470 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "6f2648e70c16418b949bde8d7ffffd2a"
format_stamp: "Formatted at 2026-08-09 07:58:22 on dist-test-slave-bw88"
I20260809 07:58:22.031344 26470 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:22.049325 26470 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:22.053103 26470 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:22.053464 26470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:22.073313 26470 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:34611
I20260809 07:58:22.073396 26564 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:34611 every 8 connection(s)
I20260809 07:58:22.074872 26470 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:22.079178 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:22.080785 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26470
I20260809 07:58:22.081010 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:22.091253 26565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a: Bootstrap starting.
I20260809 07:58:22.095786 26565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a: Neither blocks nor log segments found. Creating new log.
Aug 09 07:58:22 dist-test-slave-bw88 krb5kdc[26444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:22.096921 26565 log.cc:826] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:22.100409 26565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a: No bootstrap required, opened a new log
I20260809 07:58:22.108014 26565 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2648e70c16418b949bde8d7ffffd2a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34611 } }
I20260809 07:58:22.108361 26565 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:22.108441 26565 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2648e70c16418b949bde8d7ffffd2a, State: Initialized, Role: FOLLOWER
I20260809 07:58:22.108976 26565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2648e70c16418b949bde8d7ffffd2a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34611 } }
I20260809 07:58:22.109165 26565 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:22.109258 26565 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:22.109458 26565 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:22.111500 26565 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2648e70c16418b949bde8d7ffffd2a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34611 } }
I20260809 07:58:22.112089 26565 leader_election.cc:304] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2648e70c16418b949bde8d7ffffd2a; no voters:
I20260809 07:58:22.112733 26565 leader_election.cc:290] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:22.112869 26570 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:22.113621 26570 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [term 1 LEADER]: Becoming Leader. State: Replica: 6f2648e70c16418b949bde8d7ffffd2a, State: Running, Role: LEADER
I20260809 07:58:22.114151 26570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2648e70c16418b949bde8d7ffffd2a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34611 } }
I20260809 07:58:22.114948 26565 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:22.117247 26571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f2648e70c16418b949bde8d7ffffd2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2648e70c16418b949bde8d7ffffd2a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34611 } } }
I20260809 07:58:22.117252 26572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2648e70c16418b949bde8d7ffffd2a. Latest consensus state: current_term: 1 leader_uuid: "6f2648e70c16418b949bde8d7ffffd2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2648e70c16418b949bde8d7ffffd2a" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 34611 } } }
I20260809 07:58:22.117619 26571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:22.117713 26572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:22.120759 26575 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:22.127153 26575 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:22.138024 26575 catalog_manager.cc:1383] Generated new cluster ID: fde00ba454d441afb9eaeaaca39b9f91
I20260809 07:58:22.138154 26575 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:22.152344 26575 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:22.154100 26575 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:22.169785 26575 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 6f2648e70c16418b949bde8d7ffffd2a: Generated new TSK 0
I20260809 07:58:22.170941 26575 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:22.327338 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:34611
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:39515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:22.688054 26595 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:22.688447 26595 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:22.688613 26595 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:22.696900 26595 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:22.697139 26595 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:22.707312 26595 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:39515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:34611
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:22.708848 26595 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:22.710224 26595 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:22.720216 26604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:22.720230 26603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:22.720850 26607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:22.721433 26595 server_base.cc:1061] running on GCE node
I20260809 07:58:22.722041 26595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:22.722967 26595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:22.724203 26595 hybrid_clock.cc:648] HybridClock initialized: now 1786262302724156 us; error 59 us; skew 500 ppm
I20260809 07:58:22.724534 26595 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:22 dist-test-slave-bw88 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262302, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:22.727751 26595 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:22.729300 26595 webserver.cc:533] Webserver started at http://127.26.37.1:43713/ using document root <none> and password file <none>
I20260809 07:58:22.730067 26595 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:22.730226 26595 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:22.730594 26595 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:22.732933 26595 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "51dff7743cce4898b4ec503b83658b49"
format_stamp: "Formatted at 2026-08-09 07:58:22 on dist-test-slave-bw88"
I20260809 07:58:22.733778 26595 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "51dff7743cce4898b4ec503b83658b49"
format_stamp: "Formatted at 2026-08-09 07:58:22 on dist-test-slave-bw88"
I20260809 07:58:22.739105 26595 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:58:22.742420 26614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:22.743660 26595 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:58:22.743839 26595 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "51dff7743cce4898b4ec503b83658b49"
format_stamp: "Formatted at 2026-08-09 07:58:22 on dist-test-slave-bw88"
I20260809 07:58:22.744050 26595 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:22.786474 26595 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:22.790190 26595 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:22.790501 26595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:22.791474 26595 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:22.793155 26595 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:22.793265 26595 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:22.793447 26595 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:22.793527 26595 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:22.828783 26595 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:44643
I20260809 07:58:22.828869 26757 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:44643 every 8 connection(s)
I20260809 07:58:22.830282 26595 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:22.832167 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26595
I20260809 07:58:22.832358 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:22 dist-test-slave-bw88 krb5kdc[26444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262302, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:22.851512 26758 heartbeater.cc:344] Connected to a master server at 127.26.37.62:34611
I20260809 07:58:22.851835 26758 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:22.852604 26758 heartbeater.cc:507] Master 127.26.37.62:34611 requested a full tablet report, sending...
I20260809 07:58:22.854813 26516 ts_manager.cc:194] Registered new tserver with Master: 51dff7743cce4898b4ec503b83658b49 (127.26.37.1:44643)
I20260809 07:58:22.856351 26516 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:43775
I20260809 07:58:22.860967 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:22 dist-test-slave-bw88 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:22 dist-test-slave-bw88 krb5kdc[26444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:22.933542 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26595
Aug 09 07:58:22 dist-test-slave-bw88 krb5kdc[26444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:22.950241 26752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:23.034552 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26595
I20260809 07:58:23.058061 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26470
I20260809 07:58:23.059324 26559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:23.168633 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26470
2026-08-09T07:58:23Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/7 (4239 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/8
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:23 dist-test-slave-bw88 krb5kdc[26784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:23 dist-test-slave-bw88 krb5kdc[26784](info): set up 2 sockets
Aug 09 07:58:23 dist-test-slave-bw88 krb5kdc[26784](info): commencing operation
krb5kdc: starting...
W20260809 07:58:25.414821 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.196s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:25 dist-test-slave-bw88 krb5kdc[26784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:25Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:25.793802 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33797
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:33797
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:26.137530 26807 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:26.137866 26807 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:26.138015 26807 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:26.146329 26807 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:26.146466 26807 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:26.146602 26807 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:26.146689 26807 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:26.157193 26807 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:33797
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:33797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:26.158679 26807 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:26.160090 26807 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:26.168224 26812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:26.168310 26813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:26.168828 26816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:26.169553 26807 server_base.cc:1061] running on GCE node
I20260809 07:58:26.170231 26807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:26.171564 26807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:26.172796 26807 hybrid_clock.cc:648] HybridClock initialized: now 1786262306172755 us; error 54 us; skew 500 ppm
I20260809 07:58:26.173144 26807 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:26 dist-test-slave-bw88 krb5kdc[26784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262306, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:26.176218 26807 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:26.177809 26807 webserver.cc:533] Webserver started at http://127.26.37.62:33459/ using document root <none> and password file <none>
I20260809 07:58:26.178519 26807 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:26.178654 26807 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:26.179013 26807 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:26.181416 26807 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "79c6df38f44c4661981f7ee194e34fd4"
format_stamp: "Formatted at 2026-08-09 07:58:26 on dist-test-slave-bw88"
I20260809 07:58:26.182235 26807 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "79c6df38f44c4661981f7ee194e34fd4"
format_stamp: "Formatted at 2026-08-09 07:58:26 on dist-test-slave-bw88"
I20260809 07:58:26.187310 26807 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 07:58:26.190752 26826 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:26.192003 26807 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 07:58:26.192188 26807 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "79c6df38f44c4661981f7ee194e34fd4"
format_stamp: "Formatted at 2026-08-09 07:58:26 on dist-test-slave-bw88"
I20260809 07:58:26.192364 26807 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:26.223250 26807 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:26.231268 26807 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:26.231547 26807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:26.252254 26807 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:33797
I20260809 07:58:26.252333 26883 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:33797 every 8 connection(s)
I20260809 07:58:26.253806 26807 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:26.258222 26884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:26.262034 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26807
I20260809 07:58:26.262230 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:26.270408 26884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4: Bootstrap starting.
I20260809 07:58:26.273629 26884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:26.274644 26884 log.cc:826] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:58:26 dist-test-slave-bw88 krb5kdc[26784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:26.277583 26884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4: No bootstrap required, opened a new log
I20260809 07:58:26.283946 26884 raft_consensus.cc:359] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c6df38f44c4661981f7ee194e34fd4" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33797 } }
I20260809 07:58:26.284338 26884 raft_consensus.cc:385] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:26.284456 26884 raft_consensus.cc:740] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6df38f44c4661981f7ee194e34fd4, State: Initialized, Role: FOLLOWER
I20260809 07:58:26.285048 26884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c6df38f44c4661981f7ee194e34fd4" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33797 } }
I20260809 07:58:26.285300 26884 raft_consensus.cc:399] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:26.285470 26884 raft_consensus.cc:493] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:26.285648 26884 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:26.287698 26884 raft_consensus.cc:515] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c6df38f44c4661981f7ee194e34fd4" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33797 } }
I20260809 07:58:26.288239 26884 leader_election.cc:304] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79c6df38f44c4661981f7ee194e34fd4; no voters:
I20260809 07:58:26.288887 26884 leader_election.cc:290] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:26.289009 26890 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:26.289731 26890 raft_consensus.cc:697] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 79c6df38f44c4661981f7ee194e34fd4, State: Running, Role: LEADER
I20260809 07:58:26.290347 26890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c6df38f44c4661981f7ee194e34fd4" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33797 } }
I20260809 07:58:26.291144 26884 sys_catalog.cc:565] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:26.293617 26891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79c6df38f44c4661981f7ee194e34fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c6df38f44c4661981f7ee194e34fd4" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33797 } } }
I20260809 07:58:26.293963 26891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:26.294623 26892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79c6df38f44c4661981f7ee194e34fd4. Latest consensus state: current_term: 1 leader_uuid: "79c6df38f44c4661981f7ee194e34fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c6df38f44c4661981f7ee194e34fd4" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 33797 } } }
I20260809 07:58:26.295207 26892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:26.296418 26899 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:26.301678 26899 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:26.312537 26899 catalog_manager.cc:1383] Generated new cluster ID: 84181e938a7842428dbe73a8ae1fcd90
I20260809 07:58:26.312697 26899 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:26.324594 26899 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:26.325917 26899 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:26.334244 26899 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 79c6df38f44c4661981f7ee194e34fd4: Generated new TSK 0
I20260809 07:58:26.335013 26899 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:26.488171 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:33797
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:26.833981 26920 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:26.834288 26920 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:26.834467 26920 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:26.842638 26920 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:26.842864 26920 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:26.852597 26920 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:33797
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:26.854122 26920 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:26.855433 26920 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:26.864373 26927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:26.864403 26928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:26.865721 26930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:26.866675 26920 server_base.cc:1061] running on GCE node
I20260809 07:58:26.867187 26920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:26.868080 26920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:26.869315 26920 hybrid_clock.cc:648] HybridClock initialized: now 1786262306869275 us; error 51 us; skew 500 ppm
I20260809 07:58:26.869678 26920 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:26 dist-test-slave-bw88 krb5kdc[26784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262306, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:26.872666 26920 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:26.874444 26920 webserver.cc:533] Webserver started at http://127.26.37.1:34625/ using document root <none> and password file <none>
I20260809 07:58:26.875159 26920 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:26.875293 26920 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:26.875643 26920 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:26.878069 26920 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "5125f45c160a44e7a6ab40e131ceb7bb"
format_stamp: "Formatted at 2026-08-09 07:58:26 on dist-test-slave-bw88"
I20260809 07:58:26.878822 26920 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "5125f45c160a44e7a6ab40e131ceb7bb"
format_stamp: "Formatted at 2026-08-09 07:58:26 on dist-test-slave-bw88"
I20260809 07:58:26.883865 26920 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:58:26.887241 26939 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:26.888500 26920 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 07:58:26.888671 26920 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "5125f45c160a44e7a6ab40e131ceb7bb"
format_stamp: "Formatted at 2026-08-09 07:58:26 on dist-test-slave-bw88"
I20260809 07:58:26.888867 26920 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:26.913200 26920 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:26.916641 26920 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:26.916919 26920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:26.917884 26920 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:26.919466 26920 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:26.919588 26920 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:26.919736 26920 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:26.919807 26920 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:26.954864 26920 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:34417
I20260809 07:58:26.954982 27095 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:34417 every 8 connection(s)
I20260809 07:58:26.956338 26920 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:26.958225 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 26920
I20260809 07:58:26.958426 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:26 dist-test-slave-bw88 krb5kdc[26784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262306, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:26.976933 27096 heartbeater.cc:344] Connected to a master server at 127.26.37.62:33797
I20260809 07:58:26.977272 27096 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:26.978091 27096 heartbeater.cc:507] Master 127.26.37.62:33797 requested a full tablet report, sending...
I20260809 07:58:26.980103 26845 ts_manager.cc:194] Registered new tserver with Master: 5125f45c160a44e7a6ab40e131ceb7bb (127.26.37.1:34417)
I20260809 07:58:26.981729 26845 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:52267
I20260809 07:58:26.986486 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:27 dist-test-slave-bw88 krb5kdc[26784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:27 dist-test-slave-bw88 krb5kdc[26784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:27.055375 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26920
Aug 09 07:58:27 dist-test-slave-bw88 krb5kdc[26784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:27.072130 27090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:27.161257 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26920
I20260809 07:58:27.185467 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 26807
I20260809 07:58:27.186744 26878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:27.288399 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 26807
2026-08-09T07:58:27Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/8 (4117 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/9
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:27 dist-test-slave-bw88 krb5kdc[27113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:27 dist-test-slave-bw88 krb5kdc[27113](info): set up 2 sockets
Aug 09 07:58:27 dist-test-slave-bw88 krb5kdc[27113](info): commencing operation
krb5kdc: starting...
W20260809 07:58:29.516371 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.180s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:29 dist-test-slave-bw88 krb5kdc[27113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:29Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:29.920620 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:37407
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:37407
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:30.281263 27137 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:30.281672 27137 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:30.281775 27137 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:30.289801 27137 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:30.289889 27137 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:30.289955 27137 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:30.290012 27137 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:30.300540 27137 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:37407
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:37407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:30.302145 27137 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:30.303553 27137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:30.312284 27144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:30.312332 27145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:30.313305 27149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:30.313788 27137 server_base.cc:1061] running on GCE node
I20260809 07:58:30.314613 27137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:30.316026 27137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:30.317281 27137 hybrid_clock.cc:648] HybridClock initialized: now 1786262310317236 us; error 29 us; skew 500 ppm
I20260809 07:58:30.317678 27137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:30 dist-test-slave-bw88 krb5kdc[27113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262310, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:30.320966 27137 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:30.322669 27137 webserver.cc:533] Webserver started at http://127.26.37.62:37015/ using document root <none> and password file <none>
I20260809 07:58:30.323379 27137 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:30.323563 27137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:30.323920 27137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:30.326334 27137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "d293deb4361941a6a6644be3e34120ca"
format_stamp: "Formatted at 2026-08-09 07:58:30 on dist-test-slave-bw88"
I20260809 07:58:30.327144 27137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "d293deb4361941a6a6644be3e34120ca"
format_stamp: "Formatted at 2026-08-09 07:58:30 on dist-test-slave-bw88"
I20260809 07:58:30.332243 27137 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:58:30.335605 27158 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:30.337278 27137 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:30.337488 27137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "d293deb4361941a6a6644be3e34120ca"
format_stamp: "Formatted at 2026-08-09 07:58:30 on dist-test-slave-bw88"
I20260809 07:58:30.337678 27137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:30.350297 27137 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:30.353839 27137 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:30.354131 27137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:30.374752 27137 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:37407
I20260809 07:58:30.374787 27226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:37407 every 8 connection(s)
I20260809 07:58:30.376237 27137 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:30.380419 27227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:30.380990 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27137
I20260809 07:58:30.381208 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:30.393682 27227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca: Bootstrap starting.
Aug 09 07:58:30 dist-test-slave-bw88 krb5kdc[27113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:30.397503 27227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:30.398613 27227 log.cc:826] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:30.401254 27227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca: No bootstrap required, opened a new log
I20260809 07:58:30.406841 27227 raft_consensus.cc:359] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d293deb4361941a6a6644be3e34120ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 37407 } }
I20260809 07:58:30.407249 27227 raft_consensus.cc:385] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:30.407372 27227 raft_consensus.cc:740] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d293deb4361941a6a6644be3e34120ca, State: Initialized, Role: FOLLOWER
I20260809 07:58:30.407851 27227 consensus_queue.cc:260] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d293deb4361941a6a6644be3e34120ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 37407 } }
I20260809 07:58:30.408020 27227 raft_consensus.cc:399] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:30.408108 27227 raft_consensus.cc:493] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:30.408221 27227 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:30.410658 27227 raft_consensus.cc:515] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d293deb4361941a6a6644be3e34120ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 37407 } }
I20260809 07:58:30.411087 27227 leader_election.cc:304] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d293deb4361941a6a6644be3e34120ca; no voters:
I20260809 07:58:30.411594 27227 leader_election.cc:290] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:30.411765 27233 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:30.412549 27233 raft_consensus.cc:697] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [term 1 LEADER]: Becoming Leader. State: Replica: d293deb4361941a6a6644be3e34120ca, State: Running, Role: LEADER
I20260809 07:58:30.413100 27233 consensus_queue.cc:237] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d293deb4361941a6a6644be3e34120ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 37407 } }
I20260809 07:58:30.413691 27227 sys_catalog.cc:565] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:30.416123 27234 sys_catalog.cc:455] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d293deb4361941a6a6644be3e34120ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d293deb4361941a6a6644be3e34120ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 37407 } } }
I20260809 07:58:30.416565 27234 sys_catalog.cc:458] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:30.416457 27236 sys_catalog.cc:455] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader d293deb4361941a6a6644be3e34120ca. Latest consensus state: current_term: 1 leader_uuid: "d293deb4361941a6a6644be3e34120ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d293deb4361941a6a6644be3e34120ca" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 37407 } } }
I20260809 07:58:30.416831 27236 sys_catalog.cc:458] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:30.419097 27242 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:30.425309 27242 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:30.434113 27242 catalog_manager.cc:1383] Generated new cluster ID: 07939c2b94ef49c0b241861ebec799c4
I20260809 07:58:30.434257 27242 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:30.447014 27242 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:30.448772 27242 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:30.464020 27242 catalog_manager.cc:6092] T 00000000000000000000000000000000 P d293deb4361941a6a6644be3e34120ca: Generated new TSK 0
I20260809 07:58:30.465139 27242 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:30.616685 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:37407
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:44751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:30.972211 27269 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:30.972525 27269 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:30.972709 27269 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:30.984923 27269 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:30.985188 27269 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:30.995728 27269 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:44751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:37407
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:30.997282 27269 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:30.998703 27269 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:31.008069 27280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:31.008033 27279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:31.010918 27269 server_base.cc:1061] running on GCE node
W20260809 07:58:31.011116 27282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:31.011722 27269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:31.012639 27269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:31.013883 27269 hybrid_clock.cc:648] HybridClock initialized: now 1786262311013836 us; error 54 us; skew 500 ppm
I20260809 07:58:31.014257 27269 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:31 dist-test-slave-bw88 krb5kdc[27113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262311, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:31.017741 27269 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:31.019457 27269 webserver.cc:533] Webserver started at http://127.26.37.1:45059/ using document root <none> and password file <none>
I20260809 07:58:31.020313 27269 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:31.020464 27269 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:31.020814 27269 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:31.023322 27269 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "af94da62bdbf4cb3adbe9befdc3029ee"
format_stamp: "Formatted at 2026-08-09 07:58:31 on dist-test-slave-bw88"
I20260809 07:58:31.024132 27269 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "af94da62bdbf4cb3adbe9befdc3029ee"
format_stamp: "Formatted at 2026-08-09 07:58:31 on dist-test-slave-bw88"
I20260809 07:58:31.029698 27269 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 07:58:31.033146 27290 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:31.034494 27269 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:31.034667 27269 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "af94da62bdbf4cb3adbe9befdc3029ee"
format_stamp: "Formatted at 2026-08-09 07:58:31 on dist-test-slave-bw88"
I20260809 07:58:31.034866 27269 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:31.052405 27269 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:31.055944 27269 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:31.056170 27269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:31.057011 27269 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:31.058526 27269 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:31.058604 27269 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:31.058684 27269 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:31.058727 27269 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:31.096709 27269 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:42333
I20260809 07:58:31.096787 27449 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:42333 every 8 connection(s)
I20260809 07:58:31.098327 27269 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:31.100080 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27269
I20260809 07:58:31.100279 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:31 dist-test-slave-bw88 krb5kdc[27113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262311, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:31.121166 27450 heartbeater.cc:344] Connected to a master server at 127.26.37.62:37407
I20260809 07:58:31.121579 27450 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:31.122387 27450 heartbeater.cc:507] Master 127.26.37.62:37407 requested a full tablet report, sending...
I20260809 07:58:31.124557 27185 ts_manager.cc:194] Registered new tserver with Master: af94da62bdbf4cb3adbe9befdc3029ee (127.26.37.1:42333)
I20260809 07:58:31.126307 27185 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:47977
I20260809 07:58:31.128711 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:31 dist-test-slave-bw88 krb5kdc[27113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:31 dist-test-slave-bw88 krb5kdc[27113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:31.203646 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27269
Aug 09 07:58:31 dist-test-slave-bw88 krb5kdc[27113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:31.221552 27444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:31.307200 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27269
I20260809 07:58:31.329936 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27137
I20260809 07:58:31.331073 27221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:31.437175 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27137
2026-08-09T07:58:31Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/9 (4152 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/10
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:31 dist-test-slave-bw88 krb5kdc[27467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:31 dist-test-slave-bw88 krb5kdc[27467](info): set up 2 sockets
Aug 09 07:58:31 dist-test-slave-bw88 krb5kdc[27467](info): commencing operation
krb5kdc: starting...
W20260809 07:58:33.679157 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.190s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:33 dist-test-slave-bw88 krb5kdc[27467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:34.069905 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38529
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:38529
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:34.422412 27491 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:34.422823 27491 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:34.422999 27491 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:34.431236 27491 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:34.431378 27491 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:34.431510 27491 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:34.431592 27491 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:34.442207 27491 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:38529
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:38529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:34.443785 27491 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:34.445227 27491 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:34.453392 27496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:34.453420 27497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:34.454402 27501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:34.455129 27491 server_base.cc:1061] running on GCE node
I20260809 07:58:34.455838 27491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:34.457221 27491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:34.458459 27491 hybrid_clock.cc:648] HybridClock initialized: now 1786262314458406 us; error 93 us; skew 500 ppm
I20260809 07:58:34.458792 27491 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:34 dist-test-slave-bw88 krb5kdc[27467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262314, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:34.461989 27491 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:34.463570 27491 webserver.cc:533] Webserver started at http://127.26.37.62:36441/ using document root <none> and password file <none>
I20260809 07:58:34.464260 27491 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:34.464396 27491 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:34.464773 27491 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:34.467177 27491 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "eb546a5243cc4e269ac402920f22cb6b"
format_stamp: "Formatted at 2026-08-09 07:58:34 on dist-test-slave-bw88"
I20260809 07:58:34.467994 27491 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "eb546a5243cc4e269ac402920f22cb6b"
format_stamp: "Formatted at 2026-08-09 07:58:34 on dist-test-slave-bw88"
I20260809 07:58:34.473491 27491 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 07:58:34.477031 27511 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:34.478346 27491 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:34.478550 27491 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "eb546a5243cc4e269ac402920f22cb6b"
format_stamp: "Formatted at 2026-08-09 07:58:34 on dist-test-slave-bw88"
I20260809 07:58:34.478754 27491 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:34.504272 27491 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:34.508039 27491 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:34.508339 27491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:34.529999 27491 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:38529
I20260809 07:58:34.530050 27576 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:38529 every 8 connection(s)
I20260809 07:58:34.531634 27491 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:34.536067 27577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:34.541545 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27491
I20260809 07:58:34.541765 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:34.546682 27577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b: Bootstrap starting.
I20260809 07:58:34.550206 27577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:34.551266 27577 log.cc:826] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:58:34 dist-test-slave-bw88 krb5kdc[27467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:34.556247 27577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b: No bootstrap required, opened a new log
I20260809 07:58:34.563135 27577 raft_consensus.cc:359] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb546a5243cc4e269ac402920f22cb6b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38529 } }
I20260809 07:58:34.563599 27577 raft_consensus.cc:385] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:34.563736 27577 raft_consensus.cc:740] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb546a5243cc4e269ac402920f22cb6b, State: Initialized, Role: FOLLOWER
I20260809 07:58:34.564379 27577 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb546a5243cc4e269ac402920f22cb6b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38529 } }
I20260809 07:58:34.564631 27577 raft_consensus.cc:399] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:34.564769 27577 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:34.564961 27577 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:34.567306 27577 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb546a5243cc4e269ac402920f22cb6b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38529 } }
I20260809 07:58:34.567821 27577 leader_election.cc:304] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb546a5243cc4e269ac402920f22cb6b; no voters:
I20260809 07:58:34.568487 27577 leader_election.cc:290] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:34.568631 27583 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:34.570659 27577 sys_catalog.cc:565] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:34.575006 27583 raft_consensus.cc:697] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [term 1 LEADER]: Becoming Leader. State: Replica: eb546a5243cc4e269ac402920f22cb6b, State: Running, Role: LEADER
I20260809 07:58:34.575881 27583 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb546a5243cc4e269ac402920f22cb6b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38529 } }
I20260809 07:58:34.579548 27585 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb546a5243cc4e269ac402920f22cb6b. Latest consensus state: current_term: 1 leader_uuid: "eb546a5243cc4e269ac402920f22cb6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb546a5243cc4e269ac402920f22cb6b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38529 } } }
I20260809 07:58:34.579572 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb546a5243cc4e269ac402920f22cb6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb546a5243cc4e269ac402920f22cb6b" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 38529 } } }
I20260809 07:58:34.579923 27585 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:34.579924 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b [sys.catalog]: This master's current role is: LEADER
W20260809 07:58:34.588295 27601 catalog_manager.cc:1594] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:58:34.588433 27601 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:58:34.588574 27605 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:34.590373 27605 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:34.598609 27605 catalog_manager.cc:1383] Generated new cluster ID: fc33e38aed824663ac3c0f7b3dd2c30c
I20260809 07:58:34.598976 27605 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:34.625636 27605 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:34.626883 27605 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:34.635180 27605 catalog_manager.cc:6092] T 00000000000000000000000000000000 P eb546a5243cc4e269ac402920f22cb6b: Generated new TSK 0
I20260809 07:58:34.636037 27605 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:34.781091 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:38529
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:35.140878 27612 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:35.141192 27612 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:35.141307 27612 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:35.149616 27612 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:35.149791 27612 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:35.159902 27612 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:38529
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:35.161496 27612 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:35.162799 27612 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:35.172240 27618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:35.172255 27617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:35.172852 27620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:35.174067 27612 server_base.cc:1061] running on GCE node
I20260809 07:58:35.174777 27612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:35.175746 27612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:35.176997 27612 hybrid_clock.cc:648] HybridClock initialized: now 1786262315176946 us; error 53 us; skew 500 ppm
I20260809 07:58:35.177373 27612 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:35 dist-test-slave-bw88 krb5kdc[27467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262315, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:35.180876 27612 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:35.182452 27612 webserver.cc:533] Webserver started at http://127.26.37.1:34623/ using document root <none> and password file <none>
I20260809 07:58:35.183127 27612 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:35.183218 27612 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:35.183573 27612 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:35.186146 27612 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "dd49b306289040daafa4d432a963ec65"
format_stamp: "Formatted at 2026-08-09 07:58:35 on dist-test-slave-bw88"
I20260809 07:58:35.186913 27612 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "dd49b306289040daafa4d432a963ec65"
format_stamp: "Formatted at 2026-08-09 07:58:35 on dist-test-slave-bw88"
I20260809 07:58:35.192215 27612 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:58:35.195559 27629 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:35.196861 27612 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:35.197057 27612 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "dd49b306289040daafa4d432a963ec65"
format_stamp: "Formatted at 2026-08-09 07:58:35 on dist-test-slave-bw88"
I20260809 07:58:35.197232 27612 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:35.229534 27612 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:35.233704 27612 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:35.234030 27612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:35.235062 27612 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:35.236680 27612 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:35.236753 27612 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:35.236924 27612 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:35.236990 27612 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:35.272409 27612 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:38509
I20260809 07:58:35.272472 27773 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:38509 every 8 connection(s)
I20260809 07:58:35.273942 27612 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:35.275807 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27612
I20260809 07:58:35.276005 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:35 dist-test-slave-bw88 krb5kdc[27467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262315, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:35.294435 27774 heartbeater.cc:344] Connected to a master server at 127.26.37.62:38529
I20260809 07:58:35.294801 27774 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:35.295610 27774 heartbeater.cc:507] Master 127.26.37.62:38529 requested a full tablet report, sending...
I20260809 07:58:35.297771 27532 ts_manager.cc:194] Registered new tserver with Master: dd49b306289040daafa4d432a963ec65 (127.26.37.1:38509)
I20260809 07:58:35.299309 27532 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:59891
I20260809 07:58:35.303929 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:35 dist-test-slave-bw88 krb5kdc[27467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:35 dist-test-slave-bw88 krb5kdc[27467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:35.371349 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27612
Aug 09 07:58:35 dist-test-slave-bw88 krb5kdc[27467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:35.386881 27768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:35.469632 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27612
I20260809 07:58:35.497052 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27491
I20260809 07:58:35.498351 27569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:35.604156 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27491
2026-08-09T07:58:35Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/10 (4163 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/11
Loading random data
Initializing database '/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:58:35 dist-test-slave-bw88 krb5kdc[27791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:58:35 dist-test-slave-bw88 krb5kdc[27791](info): set up 2 sockets
Aug 09 07:58:35 dist-test-slave-bw88 krb5kdc[27791](info): commencing operation
krb5kdc: starting...
W20260809 07:58:37.879078 26772 mini_kdc.cc:121] Time spent starting KDC: real 2.226s user 0.002s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/test-user.keytab.
Aug 09 07:58:38 dist-test-slave-bw88 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:58:38Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:38.269575 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45405
--webserver_interface=127.26.37.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:42245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:45405
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:38.619997 27811 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:38.620292 27811 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:38.620383 27811 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:38.628427 27811 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:38.628508 27811 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:38.628566 27811 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:38.628614 27811 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:58:38.638857 27811 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:45405
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:45405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:38.640280 27811 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:38.641681 27811 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:38.650151 27817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:38.650164 27816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:38.651008 27819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:38.651328 27811 server_base.cc:1061] running on GCE node
I20260809 07:58:38.652050 27811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:38.653404 27811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:38.654625 27811 hybrid_clock.cc:648] HybridClock initialized: now 1786262318654591 us; error 28 us; skew 500 ppm
I20260809 07:58:38.654945 27811 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:38 dist-test-slave-bw88 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262318, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:38.658226 27811 init.cc:377] Logged in from keytab as kudu/127.26.37.62@KRBTEST.COM (short username kudu)
I20260809 07:58:38.659734 27811 webserver.cc:533] Webserver started at http://127.26.37.62:38265/ using document root <none> and password file <none>
I20260809 07:58:38.660522 27811 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:38.660667 27811 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:38.661026 27811 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:38.663478 27811 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "034e1a08d377468288e490f0cc55c9dc"
format_stamp: "Formatted at 2026-08-09 07:58:38 on dist-test-slave-bw88"
I20260809 07:58:38.664317 27811 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "034e1a08d377468288e490f0cc55c9dc"
format_stamp: "Formatted at 2026-08-09 07:58:38 on dist-test-slave-bw88"
I20260809 07:58:38.669674 27811 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 07:58:38.672974 27828 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:38.674355 27811 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:38.674546 27811 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "034e1a08d377468288e490f0cc55c9dc"
format_stamp: "Formatted at 2026-08-09 07:58:38 on dist-test-slave-bw88"
I20260809 07:58:38.674752 27811 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:38.689890 27811 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:38.698427 27811 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:38.698704 27811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:38.718956 27811 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:45405
I20260809 07:58:38.718994 27892 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:45405 every 8 connection(s)
I20260809 07:58:38.720474 27811 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:38.724685 27893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:38.728652 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27811
I20260809 07:58:38.728876 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:38.735224 27893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc: Bootstrap starting.
I20260809 07:58:38.738590 27893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:38.739619 27893 log.cc:826] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:58:38 dist-test-slave-bw88 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:38.742774 27893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc: No bootstrap required, opened a new log
I20260809 07:58:38.748378 27893 raft_consensus.cc:359] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034e1a08d377468288e490f0cc55c9dc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45405 } }
I20260809 07:58:38.748740 27893 raft_consensus.cc:385] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:38.748878 27893 raft_consensus.cc:740] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034e1a08d377468288e490f0cc55c9dc, State: Initialized, Role: FOLLOWER
I20260809 07:58:38.749657 27893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034e1a08d377468288e490f0cc55c9dc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45405 } }
I20260809 07:58:38.749908 27893 raft_consensus.cc:399] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:58:38.750063 27893 raft_consensus.cc:493] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:58:38.750262 27893 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:38.752803 27893 raft_consensus.cc:515] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034e1a08d377468288e490f0cc55c9dc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45405 } }
I20260809 07:58:38.753244 27893 leader_election.cc:304] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 034e1a08d377468288e490f0cc55c9dc; no voters:
I20260809 07:58:38.753896 27893 leader_election.cc:290] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:58:38.754035 27901 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:38.754849 27901 raft_consensus.cc:697] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [term 1 LEADER]: Becoming Leader. State: Replica: 034e1a08d377468288e490f0cc55c9dc, State: Running, Role: LEADER
I20260809 07:58:38.755384 27901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034e1a08d377468288e490f0cc55c9dc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45405 } }
I20260809 07:58:38.756011 27893 sys_catalog.cc:565] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:58:38.758378 27903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "034e1a08d377468288e490f0cc55c9dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034e1a08d377468288e490f0cc55c9dc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45405 } } }
I20260809 07:58:38.758720 27903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:38.758760 27904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 034e1a08d377468288e490f0cc55c9dc. Latest consensus state: current_term: 1 leader_uuid: "034e1a08d377468288e490f0cc55c9dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034e1a08d377468288e490f0cc55c9dc" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 45405 } } }
I20260809 07:58:38.759174 27904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:38.760399 27910 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:38.765987 27910 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:38.774999 27910 catalog_manager.cc:1383] Generated new cluster ID: d79ab90915ee4029b95f49b230e9372d
I20260809 07:58:38.775103 27910 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:38.796979 27910 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:38.799201 27910 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:38.815078 27910 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 034e1a08d377468288e490f0cc55c9dc: Generated new TSK 0
I20260809 07:58:38.815979 27910 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.26.37.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.37.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.26.37.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab.
I20260809 07:58:39.004484 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:45405
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.26.37.20:42245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:58:39.401652 27934 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:39.401934 27934 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:39.402137 27934 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:39.410383 27934 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:39.410626 27934 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:39.420646 27934 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:42245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/krb5kdc/kudu.keytab
--principal=kudu/127.26.37.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.26.37.62:45405
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:39.422284 27934 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:39.424041 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:39.433801 27943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:39.433979 27944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:39.434676 27946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:39.435277 27934 server_base.cc:1061] running on GCE node
I20260809 07:58:39.435775 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:39.436719 27934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:39.437903 27934 hybrid_clock.cc:648] HybridClock initialized: now 1786262319437856 us; error 41 us; skew 500 ppm
I20260809 07:58:39.438241 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 07:58:39 dist-test-slave-bw88 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262319, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:58:39.441296 27934 init.cc:377] Logged in from keytab as kudu/127.26.37.1@KRBTEST.COM (short username kudu)
I20260809 07:58:39.442842 27934 webserver.cc:533] Webserver started at http://127.26.37.1:36015/ using document root <none> and password file <none>
I20260809 07:58:39.443496 27934 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:39.443614 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:39.443897 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:39.446336 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "6287743bee5841408dff5a96a6d942d7"
format_stamp: "Formatted at 2026-08-09 07:58:39 on dist-test-slave-bw88"
I20260809 07:58:39.447049 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "6287743bee5841408dff5a96a6d942d7"
format_stamp: "Formatted at 2026-08-09 07:58:39 on dist-test-slave-bw88"
I20260809 07:58:39.452231 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:58:39.455461 27953 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:39.456808 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:39.456995 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "6287743bee5841408dff5a96a6d942d7"
format_stamp: "Formatted at 2026-08-09 07:58:39 on dist-test-slave-bw88"
I20260809 07:58:39.457175 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:39.474228 27934 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:39.477710 27934 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:39.477990 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:39.478866 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:39.480463 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:39.480568 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:39.480716 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:39.480779 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:39.515712 27934 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:43997
I20260809 07:58:39.515764 28093 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:43997 every 8 connection(s)
I20260809 07:58:39.517125 27934 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:39.527453 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 27934
I20260809 07:58:39.527643 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
Aug 09 07:58:39 dist-test-slave-bw88 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262319, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.37.1@KRBTEST.COM for kudu/127.26.37.62@KRBTEST.COM
I20260809 07:58:39.537111 28094 heartbeater.cc:344] Connected to a master server at 127.26.37.62:45405
I20260809 07:58:39.537496 28094 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:39.538282 28094 heartbeater.cc:507] Master 127.26.37.62:45405 requested a full tablet report, sending...
I20260809 07:58:39.540282 27852 ts_manager.cc:194] Registered new tserver with Master: 6287743bee5841408dff5a96a6d942d7 (127.26.37.1:43997)
I20260809 07:58:39.542063 27852 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.37.1@KRBTEST.COM'} at 127.26.37.1:56359
I20260809 07:58:39.543684 26772 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 09 07:58:39 dist-test-slave-bw88 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:58:39 dist-test-slave-bw88 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.26.37.62@KRBTEST.COM
I20260809 07:58:39.611099 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27934
Aug 09 07:58:39 dist-test-slave-bw88 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786262319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.37.1@KRBTEST.COM
I20260809 07:58:39.628513 28089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:39.713508 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27934
I20260809 07:58:39.739145 26772 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu pid 27811
I20260809 07:58:39.740338 27888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 07:58:39.844218 26772 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu with pid 27811
2026-08-09T07:58:39Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/11 (4243 ms)
[----------] 37 tests from ExternalMiniClusterTest (631176 ms total)
[----------] 1 test from Kudu3762Test
[ RUN ] Kudu3762Test.AddMaster
2026-08-09T07:58:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:58:39Z Disabled control of system clock
I20260809 07:58:39.948208 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:44045
--webserver_interface=127.26.37.62
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:44045,127.26.37.61:46239
--flush_threshold_mb=0
--flush_threshold_secs=1
--log_cache_size_limit_mb=1
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_segment_size_mb=1
--maintenance_manager_num_threads=4
--maintenance_manager_polling_interval_ms=10
--unlock_experimental_flags=true with env {}
W20260809 07:58:40.299952 28113 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:40.300243 28113 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:40.300323 28113 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:40.308303 28113 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260809 07:58:40.308380 28113 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260809 07:58:40.308451 28113 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:40.308499 28113 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:40.308553 28113 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:40.308598 28113 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260809 07:58:40.308682 28113 flags.cc:432] Enabled experimental flag: --flush_threshold_mb=0
W20260809 07:58:40.308723 28113 flags.cc:432] Enabled experimental flag: --flush_threshold_secs=1
I20260809 07:58:40.318059 28113 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:44045,127.26.37.61:46239
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.62:44045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.37.62
--webserver_port=0
--flush_threshold_mb=0
--flush_threshold_secs=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--maintenance_manager_num_threads=4
--maintenance_manager_polling_interval_ms=10
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
W20260809 07:58:40.318310 28113 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.37.62:44045,127.26.37.61:46239'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 07:58:40.319567 28113 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:40.321301 28113 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:40.328963 28123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:40.329030 28122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:40.329901 28125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:40.330677 28113 server_base.cc:1061] running on GCE node
I20260809 07:58:40.331616 28113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:40.332872 28113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:40.334127 28113 hybrid_clock.cc:648] HybridClock initialized: now 1786262320334055 us; error 62 us; skew 500 ppm
I20260809 07:58:40.334436 28113 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:58:40.336597 28113 webserver.cc:533] Webserver started at http://127.26.37.62:41755/ using document root <none> and password file <none>
I20260809 07:58:40.337277 28113 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:40.337445 28113 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:40.337801 28113 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:40.340094 28113 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/data/instance:
uuid: "16fe06ae10064d4b894ac1d45c6ca3f6"
format_stamp: "Formatted at 2026-08-09 07:58:40 on dist-test-slave-bw88"
I20260809 07:58:40.340854 28113 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/wal/instance:
uuid: "16fe06ae10064d4b894ac1d45c6ca3f6"
format_stamp: "Formatted at 2026-08-09 07:58:40 on dist-test-slave-bw88"
I20260809 07:58:40.346020 28113 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260809 07:58:40.349202 28131 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:40.350507 28113 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:58:40.350709 28113 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/wal
uuid: "16fe06ae10064d4b894ac1d45c6ca3f6"
format_stamp: "Formatted at 2026-08-09 07:58:40 on dist-test-slave-bw88"
I20260809 07:58:40.350895 28113 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:40.374730 28113 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:40.375612 28113 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:40.375914 28113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:40.398123 28113 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.62:44045
I20260809 07:58:40.398146 28192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.62:44045 every 8 connection(s)
I20260809 07:58:40.399744 28113 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/data/info.pb
I20260809 07:58:40.404004 28193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:40.405828 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28113
I20260809 07:58:40.406073 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-0/wal/instance
I20260809 07:58:40.407364 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:46239
--webserver_interface=127.26.37.61
--webserver_port=0
--builtin_ntp_servers=127.26.37.20:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.37.62:44045,127.26.37.61:46239
--flush_threshold_mb=0
--flush_threshold_secs=1
--log_cache_size_limit_mb=1
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_segment_size_mb=1
--maintenance_manager_num_threads=4
--maintenance_manager_polling_interval_ms=10
--unlock_experimental_flags=true with env {}
I20260809 07:58:40.410017 28193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:58:40.425948 28193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } has no permanent_uuid. Determining permanent_uuid...
W20260809 07:58:40.427271 28134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.37.61:46239: connect: Connection refused (error 111)
W20260809 07:58:40.428993 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:46239: Network error: Client connection negotiation failed: client connection to 127.26.37.61:46239: connect: Connection refused (error 111)
I20260809 07:58:40.478315 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } attempt: 1
W20260809 07:58:40.480459 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:46239: Network error: Client connection negotiation failed: client connection to 127.26.37.61:46239: connect: Connection refused (error 111)
I20260809 07:58:40.548731 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } attempt: 2
W20260809 07:58:40.551213 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:46239: Network error: Client connection negotiation failed: client connection to 127.26.37.61:46239: connect: Connection refused (error 111)
I20260809 07:58:40.642495 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } attempt: 3
W20260809 07:58:40.644816 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:46239: Network error: Client connection negotiation failed: client connection to 127.26.37.61:46239: connect: Connection refused (error 111)
W20260809 07:58:40.766954 28198 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:40.767307 28198 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:40.767446 28198 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:40.775354 28198 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260809 07:58:40.775466 28198 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260809 07:58:40.775609 28198 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:58:40.775693 28198 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:40.775803 28198 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:58:40.775892 28198 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260809 07:58:40.776026 28198 flags.cc:432] Enabled experimental flag: --flush_threshold_mb=0
W20260809 07:58:40.776101 28198 flags.cc:432] Enabled experimental flag: --flush_threshold_secs=1
I20260809 07:58:40.785807 28198 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.37.62:44045,127.26.37.61:46239
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.37.61:46239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.37.61
--webserver_port=0
--flush_threshold_mb=0
--flush_threshold_secs=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--maintenance_manager_num_threads=4
--maintenance_manager_polling_interval_ms=10
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
W20260809 07:58:40.786156 28198 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.37.62:44045,127.26.37.61:46239'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 07:58:40.787479 28198 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:40.788057 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } attempt: 4
I20260809 07:58:40.789582 28198 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:40.790128 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.26.37.61:46239: Network error: Client connection negotiation failed: client connection to 127.26.37.61:46239: connect: Connection refused (error 111)
W20260809 07:58:40.797674 28209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:40.797677 28210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:40.798223 28214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:40.799096 28198 server_base.cc:1061] running on GCE node
I20260809 07:58:40.799803 28198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:40.801193 28198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:40.802440 28198 hybrid_clock.cc:648] HybridClock initialized: now 1786262320802372 us; error 67 us; skew 500 ppm
I20260809 07:58:40.802790 28198 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:58:40.804888 28198 webserver.cc:533] Webserver started at http://127.26.37.61:35367/ using document root <none> and password file <none>
I20260809 07:58:40.805730 28198 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:40.805884 28198 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:40.806246 28198 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:40.808985 28198 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/data/instance:
uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a"
format_stamp: "Formatted at 2026-08-09 07:58:40 on dist-test-slave-bw88"
I20260809 07:58:40.809796 28198 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/wal/instance:
uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a"
format_stamp: "Formatted at 2026-08-09 07:58:40 on dist-test-slave-bw88"
I20260809 07:58:40.815021 28198 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:58:40.818331 28224 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:40.819547 28198 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:58:40.819734 28198 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/wal
uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a"
format_stamp: "Formatted at 2026-08-09 07:58:40 on dist-test-slave-bw88"
I20260809 07:58:40.819921 28198 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:40.836500 28198 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:40.837265 28198 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:40.837692 28198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:40.859019 28198 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.61:46239
I20260809 07:58:40.859045 28295 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.61:46239 every 8 connection(s)
I20260809 07:58:40.860615 28198 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/data/info.pb
I20260809 07:58:40.864825 28296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:40.864945 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28198
I20260809 07:58:40.865190 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/master-1/wal/instance
I20260809 07:58:40.872399 28296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:58:40.882829 28296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:58:40.892189 28296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: Bootstrap starting.
I20260809 07:58:40.895208 28296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:40.896433 28296 log.cc:826] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:40.898869 28296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: No bootstrap required, opened a new log
I20260809 07:58:40.903888 28296 raft_consensus.cc:359] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } }
I20260809 07:58:40.904234 28296 raft_consensus.cc:385] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:40.904317 28296 raft_consensus.cc:740] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e6b2cd9629422c8bcfa82f3cbad53a, State: Initialized, Role: FOLLOWER
I20260809 07:58:40.904866 28296 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } }
I20260809 07:58:40.905663 28305 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } } }
I20260809 07:58:40.905951 28305 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:58:40.906692 28296 sys_catalog.cc:565] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:58:40.911334 28236 tablet.cc:2406] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:58:40.916220 28318 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:58:40.916365 28318 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:58:40.913865 28198 master_runner.cc:187] Error getting master registration for 127.26.37.62:44045: OK, instance_id { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" instance_seqno: 1786262320392535 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:58:40.918426 28198 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:58:41.089385 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } attempt: 5
I20260809 07:58:41.100874 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Bootstrap starting.
I20260809 07:58:41.104105 28193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:41.105195 28193 log.cc:826] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:41.107764 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: No bootstrap required, opened a new log
I20260809 07:58:41.112875 28193 raft_consensus.cc:359] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } }
I20260809 07:58:41.113128 28193 raft_consensus.cc:385] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:41.113281 28193 raft_consensus.cc:740] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fe06ae10064d4b894ac1d45c6ca3f6, State: Initialized, Role: FOLLOWER
I20260809 07:58:41.113916 28193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } }
I20260809 07:58:41.114869 28321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } } }
I20260809 07:58:41.115149 28321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:58:41.115959 28193 sys_catalog.cc:565] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:58:41.121116 28140 tablet.cc:2406] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:58:41.126763 28336 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:58:41.126950 28336 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:58:41.128445 28113 master_runner.cc:264] No leader master found from master 16fe06ae10064d4b894ac1d45c6ca3f6
I20260809 07:58:41.128613 28113 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:58:41.311887 28321 raft_consensus.cc:493] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:41.312183 28321 raft_consensus.cc:515] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } }
I20260809 07:58:41.313642 28321 leader_election.cc:290] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e6b2cd9629422c8bcfa82f3cbad53a (127.26.37.61:46239)
I20260809 07:58:41.314316 28263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" is_pre_election: true
I20260809 07:58:41.314754 28263 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16fe06ae10064d4b894ac1d45c6ca3f6 in term 0.
I20260809 07:58:41.315361 28134 leader_election.cc:304] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 16fe06ae10064d4b894ac1d45c6ca3f6, d2e6b2cd9629422c8bcfa82f3cbad53a; no voters:
I20260809 07:58:41.315723 28321 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:41.315891 28321 raft_consensus.cc:493] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:41.316036 28321 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:41.318584 28321 raft_consensus.cc:515] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } }
I20260809 07:58:41.319137 28321 leader_election.cc:290] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [CANDIDATE]: Term 1 election: Requested vote from peers d2e6b2cd9629422c8bcfa82f3cbad53a (127.26.37.61:46239)
I20260809 07:58:41.319692 28263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a"
I20260809 07:58:41.319855 28263 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:41.322120 28263 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fe06ae10064d4b894ac1d45c6ca3f6 in term 1.
I20260809 07:58:41.322577 28134 leader_election.cc:304] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 16fe06ae10064d4b894ac1d45c6ca3f6, d2e6b2cd9629422c8bcfa82f3cbad53a; no voters:
I20260809 07:58:41.322896 28321 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:41.323421 28321 raft_consensus.cc:697] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [term 1 LEADER]: Becoming Leader. State: Replica: 16fe06ae10064d4b894ac1d45c6ca3f6, State: Running, Role: LEADER
I20260809 07:58:41.323930 28321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } }
I20260809 07:58:41.327348 28341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16fe06ae10064d4b894ac1d45c6ca3f6. Latest consensus state: current_term: 1 leader_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } } }
I20260809 07:58:41.327643 28341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:41.328553 28343 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260809 07:58:41.330951 28343 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260809 07:58:41.339489 28263 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [term 1 FOLLOWER]: Refusing update from remote peer 16fe06ae10064d4b894ac1d45c6ca3f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:58:41.340348 28341 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:41.346797 28305 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16fe06ae10064d4b894ac1d45c6ca3f6. Latest consensus state: current_term: 1 leader_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } } }
I20260809 07:58:41.346992 28305 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:58:41.349870 28341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } } }
I20260809 07:58:41.350070 28341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:58:41.351644 28343 catalog_manager.cc:1383] Generated new cluster ID: 4b8bba071d104a73ae592f31d553cf0b
I20260809 07:58:41.351755 28343 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260809 07:58:41.352349 28305 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fe06ae10064d4b894ac1d45c6ca3f6" member_type: VOTER last_known_addr { host: "127.26.37.62" port: 44045 } } peers { permanent_uuid: "d2e6b2cd9629422c8bcfa82f3cbad53a" member_type: VOTER last_known_addr { host: "127.26.37.61" port: 46239 } } }
I20260809 07:58:41.352525 28305 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:58:41.353677 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:41.357479 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:41.417948 28343 catalog_manager.cc:1406] Generated new certificate authority record
I20260809 07:58:41.419797 28343 catalog_manager.cc:1540] Loading token signing keys...
I20260809 07:58:41.418888 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.064s user 0.032s sys 0.013s Metrics: {"bytes_written":4776,"cfile_init":1,"dirs.queue_time_us":311,"dirs.run_cpu_time_us":276,"dirs.run_wall_time_us":1142,"drs_written":1,"lbm_read_time_us":117,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1145,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1,"thread_start_us":203,"threads_started":1}
I20260809 07:58:41.421109 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.421522 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:41.421864 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.422277 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260809 07:58:41.422544 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:41.434391 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.011s sys 0.000s Metrics: {"bytes_written":5617,"cfile_init":1,"dirs.queue_time_us":42,"dirs.run_cpu_time_us":163,"dirs.run_wall_time_us":1332,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":889,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1}
I20260809 07:58:41.435570 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.435906 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.436272 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260809 07:58:41.436370 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:41.438274 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.080s user 0.043s sys 0.005s Metrics: {"bytes_written":4776,"cfile_init":1,"dirs.queue_time_us":11135,"dirs.run_cpu_time_us":179,"dirs.run_wall_time_us":1239,"drs_written":1,"lbm_read_time_us":109,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1110,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1,"thread_start_us":10972,"threads_started":1}
I20260809 07:58:41.439868 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.440199 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.440271 28343 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Generated new TSK 0
I20260809 07:58:41.440546 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:41.440817 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4}
I20260809 07:58:41.441280 28343 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260809 07:58:41.441252 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:41.450640 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:41.453966 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.010s sys 0.002s Metrics: {"bytes_written":5966,"cfile_init":1,"dirs.queue_time_us":60,"dirs.run_cpu_time_us":234,"dirs.run_wall_time_us":1380,"drs_written":1,"lbm_read_time_us":69,"lbm_reads_lt_1ms":4,"lbm_write_time_us":916,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260809 07:58:41.455231 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260809 07:58:41.456293 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 174 bytes on disk
I20260809 07:58:41.456895 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 174 bytes on disk
I20260809 07:58:41.457352 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 174 bytes on disk
I20260809 07:58:41.457752 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":120,"lbm_reads_lt_1ms":4}
I20260809 07:58:41.457917 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:41.458287 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:41.462508 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.009s sys 0.000s Metrics: {"bytes_written":5068,"cfile_init":1,"dirs.queue_time_us":61,"dirs.run_cpu_time_us":172,"dirs.run_wall_time_us":1168,"drs_written":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4,"lbm_write_time_us":561,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1}
I20260809 07:58:41.463259 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.463555 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.463845 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260809 07:58:41.464052 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":135,"lbm_reads_lt_1ms":4}
I20260809 07:58:41.464381 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:41.464579 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:41.471349 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.1:0
--local_ip_for_outbound_sockets=127.26.37.1
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:44045,127.26.37.61:46239
--builtin_ntp_servers=127.26.37.20:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:58:41.479341 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260809 07:58:41.480247 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4}
I20260809 07:58:41.483026 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.022s sys 0.004s Metrics: {"bytes_written":6026,"cfile_cache_hit":12,"cfile_cache_hit_bytes":1552,"cfile_cache_miss":28,"cfile_cache_miss_bytes":2194,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1830,"dirs.run_cpu_time_us":906,"dirs.run_wall_time_us":5184,"drs_written":1,"lbm_read_time_us":928,"lbm_reads_lt_1ms":56,"lbm_write_time_us":816,"lbm_writes_lt_1ms":27,"mutex_wait_us":161,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":3,"thread_start_us":3379,"threads_started":3}
W20260809 07:58:41.834678 28357 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:41.834962 28357 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:41.835038 28357 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:41.842934 28357 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:41.843073 28357 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.1
I20260809 07:58:41.852293 28357 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.37.1
--webserver_port=0
--tserver_master_addrs=127.26.37.62:44045,127.26.37.61:46239
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.1
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:41.853796 28357 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:41.855592 28357 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:41.864714 28367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:41.864738 28363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:41.867489 28369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:41.867664 28357 server_base.cc:1061] running on GCE node
I20260809 07:58:41.868278 28357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:41.869320 28357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:41.870577 28357 hybrid_clock.cc:648] HybridClock initialized: now 1786262321870505 us; error 59 us; skew 500 ppm
I20260809 07:58:41.870906 28357 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:58:41.873291 28357 webserver.cc:533] Webserver started at http://127.26.37.1:39611/ using document root <none> and password file <none>
I20260809 07:58:41.874174 28357 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:41.874305 28357 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:41.874750 28357 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:41.877756 28357 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/data/instance:
uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
format_stamp: "Formatted at 2026-08-09 07:58:41 on dist-test-slave-bw88"
I20260809 07:58:41.878711 28357 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance:
uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
format_stamp: "Formatted at 2026-08-09 07:58:41 on dist-test-slave-bw88"
I20260809 07:58:41.884748 28357 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 07:58:41.888055 28376 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:41.889452 28357 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:58:41.889636 28357 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal
uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
format_stamp: "Formatted at 2026-08-09 07:58:41 on dist-test-slave-bw88"
I20260809 07:58:41.889827 28357 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:41.911047 28357 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:41.911777 28357 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:41.912029 28357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:41.912863 28357 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:41.914414 28357 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:41.914558 28357 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:41.914716 28357 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:41.914816 28357 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:41.923084 28318 catalog_manager.cc:1295] Loaded cluster ID: 4b8bba071d104a73ae592f31d553cf0b
I20260809 07:58:41.923235 28318 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: loading cluster ID for follower catalog manager: success
I20260809 07:58:41.928514 28318 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: acquiring CA information for follower catalog manager: success
I20260809 07:58:41.932548 28318 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:58:41.951682 28357 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.1:39223
I20260809 07:58:41.951793 28524 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.1:39223 every 8 connection(s)
I20260809 07:58:41.953059 28357 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/data/info.pb
I20260809 07:58:41.957904 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28357
I20260809 07:58:41.958166 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/instance
I20260809 07:58:41.963378 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.2:0
--local_ip_for_outbound_sockets=127.26.37.2
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:44045,127.26.37.61:46239
--builtin_ntp_servers=127.26.37.20:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:58:41.977115 28525 heartbeater.cc:344] Connected to a master server at 127.26.37.61:46239
I20260809 07:58:41.977522 28525 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:41.978538 28525 heartbeater.cc:507] Master 127.26.37.61:46239 requested a full tablet report, sending...
I20260809 07:58:41.981505 28250 ts_manager.cc:194] Registered new tserver with Master: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223)
I20260809 07:58:41.981774 28526 heartbeater.cc:344] Connected to a master server at 127.26.37.62:44045
I20260809 07:58:41.981906 28526 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:41.982467 28526 heartbeater.cc:507] Master 127.26.37.62:44045 requested a full tablet report, sending...
I20260809 07:58:41.984195 28153 ts_manager.cc:194] Registered new tserver with Master: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223)
I20260809 07:58:41.985708 28153 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.1:35945
W20260809 07:58:42.328709 28536 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:42.329022 28536 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:42.329164 28536 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:42.336954 28536 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:42.337168 28536 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.2
I20260809 07:58:42.346776 28536 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.37.2
--webserver_port=0
--tserver_master_addrs=127.26.37.62:44045,127.26.37.61:46239
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.2
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:42.348276 28536 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:42.350158 28536 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:42.360049 28544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:42.360304 28545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:42.360891 28551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:42.361634 28536 server_base.cc:1061] running on GCE node
I20260809 07:58:42.362227 28536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:42.363168 28536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:42.364409 28536 hybrid_clock.cc:648] HybridClock initialized: now 1786262322364351 us; error 37 us; skew 500 ppm
I20260809 07:58:42.364787 28536 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:58:42.366914 28536 webserver.cc:533] Webserver started at http://127.26.37.2:34989/ using document root <none> and password file <none>
I20260809 07:58:42.367699 28536 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:42.367892 28536 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:42.368269 28536 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:42.370674 28536 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/data/instance:
uuid: "24137fd377124538a31e92f3b6df0673"
format_stamp: "Formatted at 2026-08-09 07:58:42 on dist-test-slave-bw88"
I20260809 07:58:42.371488 28536 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance:
uuid: "24137fd377124538a31e92f3b6df0673"
format_stamp: "Formatted at 2026-08-09 07:58:42 on dist-test-slave-bw88"
I20260809 07:58:42.376796 28536 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 07:58:42.380370 28562 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:42.381798 28536 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:58:42.381974 28536 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal
uuid: "24137fd377124538a31e92f3b6df0673"
format_stamp: "Formatted at 2026-08-09 07:58:42 on dist-test-slave-bw88"
I20260809 07:58:42.382203 28536 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:42.424229 28536 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:42.425225 28536 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:42.425617 28536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:42.426954 28536 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:42.428702 28536 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:42.428789 28536 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:42.428872 28536 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:42.428926 28536 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:42.465181 28536 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.2:38379
I20260809 07:58:42.465283 28709 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.2:38379 every 8 connection(s)
I20260809 07:58:42.466722 28536 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/data/info.pb
I20260809 07:58:42.473811 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28536
I20260809 07:58:42.474027 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/instance
I20260809 07:58:42.478220 26772 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
/tmp/dist-test-tasktiCCAC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.37.3:0
--local_ip_for_outbound_sockets=127.26.37.3
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:44045,127.26.37.61:46239
--builtin_ntp_servers=127.26.37.20:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260809 07:58:42.482810 28712 heartbeater.cc:344] Connected to a master server at 127.26.37.62:44045
I20260809 07:58:42.483193 28712 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:42.484010 28712 heartbeater.cc:507] Master 127.26.37.62:44045 requested a full tablet report, sending...
I20260809 07:58:42.484939 28710 heartbeater.cc:344] Connected to a master server at 127.26.37.61:46239
I20260809 07:58:42.485082 28710 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:42.485625 28710 heartbeater.cc:507] Master 127.26.37.61:46239 requested a full tablet report, sending...
I20260809 07:58:42.485791 28153 ts_manager.cc:194] Registered new tserver with Master: 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:42.486704 28153 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.2:46119
I20260809 07:58:42.486760 28250 ts_manager.cc:194] Registered new tserver with Master: 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
W20260809 07:58:42.845889 28720 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:58:42.846269 28720 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:58:42.846422 28720 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260809 07:58:42.854372 28720 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:58:42.854564 28720 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.37.3
I20260809 07:58:42.864269 28720 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.37.20:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.37.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.37.3
--webserver_port=0
--tserver_master_addrs=127.26.37.62:44045,127.26.37.61:46239
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.37.3
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 4bcd631c449d3c6649730ca6e07660c8168f0d44
build type FASTDEBUG
built by None at 09 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13254
ASAN enabled
I20260809 07:58:42.865810 28720 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:58:42.867697 28720 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260809 07:58:42.877345 28727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:42.877391 28726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:58:42.877849 28729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:58:42.878640 28720 server_base.cc:1061] running on GCE node
I20260809 07:58:42.879112 28720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:58:42.879906 28720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:58:42.881101 28720 hybrid_clock.cc:648] HybridClock initialized: now 1786262322881054 us; error 38 us; skew 500 ppm
I20260809 07:58:42.881513 28720 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:58:42.883708 28720 webserver.cc:533] Webserver started at http://127.26.37.3:46667/ using document root <none> and password file <none>
I20260809 07:58:42.884409 28720 fs_manager.cc:362] Metadata directory not provided
I20260809 07:58:42.884542 28720 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:58:42.884884 28720 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:58:42.887537 28720 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/data/instance:
uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
format_stamp: "Formatted at 2026-08-09 07:58:42 on dist-test-slave-bw88"
I20260809 07:58:42.888348 28720 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance:
uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
format_stamp: "Formatted at 2026-08-09 07:58:42 on dist-test-slave-bw88"
I20260809 07:58:42.893934 28720 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 07:58:42.897279 28736 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:42.898561 28720 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:58:42.898731 28720 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal
uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
format_stamp: "Formatted at 2026-08-09 07:58:42 on dist-test-slave-bw88"
I20260809 07:58:42.898934 28720 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:58:42.918207 28720 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260809 07:58:42.918902 28720 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:58:42.919191 28720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:58:42.920111 28720 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260809 07:58:42.921833 28720 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:58:42.921960 28720 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:42.922079 28720 ts_tablet_manager.cc:616] Registered 0 tablets
I20260809 07:58:42.922168 28720 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:58:42.959585 28720 rpc_server.cc:307] RPC server started. Bound to: 127.26.37.3:38449
I20260809 07:58:42.959623 28885 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.37.3:38449 every 8 connection(s)
I20260809 07:58:42.960961 28720 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/data/info.pb
I20260809 07:58:42.965719 26772 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasktiCCAC/build/asan/bin/kudu as pid 28720
I20260809 07:58:42.965937 26772 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/instance
I20260809 07:58:42.977063 28888 heartbeater.cc:344] Connected to a master server at 127.26.37.61:46239
I20260809 07:58:42.977461 28888 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:42.977625 28890 heartbeater.cc:344] Connected to a master server at 127.26.37.62:44045
I20260809 07:58:42.977771 28890 heartbeater.cc:461] Registering TS with master...
I20260809 07:58:42.978590 28890 heartbeater.cc:507] Master 127.26.37.62:44045 requested a full tablet report, sending...
I20260809 07:58:42.978610 28888 heartbeater.cc:507] Master 127.26.37.61:46239 requested a full tablet report, sending...
I20260809 07:58:42.980232 28153 ts_manager.cc:194] Registered new tserver with Master: e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:42.980392 28250 ts_manager.cc:194] Registered new tserver with Master: e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:42.981240 28153 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.26.37.3:40839
I20260809 07:58:42.986900 26772 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260809 07:58:42.988497 28526 heartbeater.cc:499] Master 127.26.37.62:44045 was elected leader, sending a full tablet report...
I20260809 07:58:43.012645 28153 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51156:
name: "table-0"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
W20260809 07:58:43.018092 28153 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:58:43.031181 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.036545 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.052436 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:43.053107 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260809 07:58:43.051208 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.008s sys 0.008s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":4984,"dirs.run_cpu_time_us":193,"dirs.run_wall_time_us":4748,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":677,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17,"thread_start_us":5207,"threads_started":1}
I20260809 07:58:43.054915 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.032994
I20260809 07:58:43.063879 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.064203 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.068864 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.032994
I20260809 07:58:43.069232 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.072427 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.003s sys 0.002s Metrics: {"bytes_written":697,"delete_count":0,"lbm_write_time_us":137,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":16}
I20260809 07:58:43.092057 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.055s user 0.010s sys 0.012s Metrics: {"bytes_written":697,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":32040,"dirs.run_cpu_time_us":164,"dirs.run_wall_time_us":6785,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1170,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":17,"thread_start_us":32135,"threads_started":1,"update_count":16}
I20260809 07:58:43.092918 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:43.106913 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.349724
I20260809 07:58:43.107242 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:43.107575 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:43.108155 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4}
I20260809 07:58:43.112664 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.112962 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.113476 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.349724
I20260809 07:58:43.113826 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.121560 28782 tablet_service.cc:1511] Processing CreateTablet for tablet b1365599d9764ccfbfb0cb954f83de69 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.117022 28441 tablet_service.cc:1511] Processing CreateTablet for tablet 9b93c5f310964747a018b899b0cb2dbf (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.125082 28782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1365599d9764ccfbfb0cb954f83de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.125146 28441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b93c5f310964747a018b899b0cb2dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.121596 28798 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac8194b8d8946d797adb804d6420646 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.129889 28798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac8194b8d8946d797adb804d6420646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.129985 28438 tablet_service.cc:1511] Processing CreateTablet for tablet acb842662fa640ee932f06fd7b5b7fc2 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.130124 28437 tablet_service.cc:1511] Processing CreateTablet for tablet 6eac9128f38545f09d093eed3c9839ae (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.130357 28438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb842662fa640ee932f06fd7b5b7fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.130590 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eac9128f38545f09d093eed3c9839ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.121853 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 841c52b2284042a8a4dab34f8ed13fe2 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.133848 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 841c52b2284042a8a4dab34f8ed13fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.121321 28789 tablet_service.cc:1511] Processing CreateTablet for tablet 9fec26011dec4e91868a1cbe71cdf371 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.135504 28789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fec26011dec4e91868a1cbe71cdf371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.135497 28435 tablet_service.cc:1511] Processing CreateTablet for tablet 03a746a63ed14fb18d0b9f21a32a40ad (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.136024 28435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a746a63ed14fb18d0b9f21a32a40ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.136646 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.017s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1677,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":738,"dirs.run_cpu_time_us":231,"dirs.run_wall_time_us":1172,"lbm_read_time_us":462,"lbm_reads_lt_1ms":21,"lbm_write_time_us":354,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":12244,"threads_started":1,"update_count":16}
I20260809 07:58:43.121905 28795 tablet_service.cc:1511] Processing CreateTablet for tablet 6eac9128f38545f09d093eed3c9839ae (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.137828 28795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eac9128f38545f09d093eed3c9839ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.122224 28785 tablet_service.cc:1511] Processing CreateTablet for tablet 5745cdf610dd4552bfd339a7ed04e95c (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.141389 28785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5745cdf610dd4552bfd339a7ed04e95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.141734 28434 tablet_service.cc:1511] Processing CreateTablet for tablet 10733adc026c49399d8d6649c5fe7705 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.142244 28434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10733adc026c49399d8d6649c5fe7705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.144860 28433 tablet_service.cc:1511] Processing CreateTablet for tablet 392548585cf1446f99dbc268831cae8e (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.145109 28431 tablet_service.cc:1511] Processing CreateTablet for tablet 9fec26011dec4e91868a1cbe71cdf371 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.145318 28433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 392548585cf1446f99dbc268831cae8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.122002 28797 tablet_service.cc:1511] Processing CreateTablet for tablet acb842662fa640ee932f06fd7b5b7fc2 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.145602 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fec26011dec4e91868a1cbe71cdf371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.122427 28794 tablet_service.cc:1511] Processing CreateTablet for tablet 03a746a63ed14fb18d0b9f21a32a40ad (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.122798 28783 tablet_service.cc:1511] Processing CreateTablet for tablet 6b81ec52183e4bd28ed4d17ed15f5414 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.122874 28791 tablet_service.cc:1511] Processing CreateTablet for tablet 392548585cf1446f99dbc268831cae8e (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.122991 28793 tablet_service.cc:1511] Processing CreateTablet for tablet 10733adc026c49399d8d6649c5fe7705 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.123113 28790 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd831b22d6044209063ec4b9a984e6d (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.116585 28439 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac8194b8d8946d797adb804d6420646 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.146919 28439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac8194b8d8946d797adb804d6420646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.123265 28788 tablet_service.cc:1511] Processing CreateTablet for tablet b95825d687ab41878ef822421d853385 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.121428 28784 tablet_service.cc:1511] Processing CreateTablet for tablet a2d60a37346b469781732b692cf55c63 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.123373 28800 tablet_service.cc:1511] Processing CreateTablet for tablet 9b93c5f310964747a018b899b0cb2dbf (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.121495 28786 tablet_service.cc:1511] Processing CreateTablet for tablet 437e82d85fc445ce8ae904c2c71d08c7 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.149925 28432 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd831b22d6044209063ec4b9a984e6d (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.150480 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd831b22d6044209063ec4b9a984e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.151515 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 5745cdf610dd4552bfd339a7ed04e95c (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.152011 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5745cdf610dd4552bfd339a7ed04e95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.153967 28429 tablet_service.cc:1511] Processing CreateTablet for tablet b95825d687ab41878ef822421d853385 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.154534 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b95825d687ab41878ef822421d853385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.154713 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 278 bytes on disk
I20260809 07:58:43.155090 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 278 bytes on disk
I20260809 07:58:43.155356 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20260809 07:58:43.155503 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.155676 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb842662fa640ee932f06fd7b5b7fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.157883 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 437e82d85fc445ce8ae904c2c71d08c7 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.158488 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 437e82d85fc445ce8ae904c2c71d08c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.161619 28794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a746a63ed14fb18d0b9f21a32a40ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.161702 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 841c52b2284042a8a4dab34f8ed13fe2 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.162333 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 6b81ec52183e4bd28ed4d17ed15f5414 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.162873 28424 tablet_service.cc:1511] Processing CreateTablet for tablet a2d60a37346b469781732b692cf55c63 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.163780 28422 tablet_service.cc:1511] Processing CreateTablet for tablet b1365599d9764ccfbfb0cb954f83de69 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.162338 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 841c52b2284042a8a4dab34f8ed13fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.164745 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b81ec52183e4bd28ed4d17ed15f5414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.165063 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1365599d9764ccfbfb0cb954f83de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.165824 28800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b93c5f310964747a018b899b0cb2dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.169620 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d60a37346b469781732b692cf55c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.170181 28791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 392548585cf1446f99dbc268831cae8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.172012 28793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10733adc026c49399d8d6649c5fe7705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.174024 28790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd831b22d6044209063ec4b9a984e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.177203 28783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b81ec52183e4bd28ed4d17ed15f5414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.180892 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b95825d687ab41878ef822421d853385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.183346 28786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 437e82d85fc445ce8ae904c2c71d08c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.183570 28930 tablet_bootstrap.cc:492] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.184211 28784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d60a37346b469781732b692cf55c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.200167 28612 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd831b22d6044209063ec4b9a984e6d (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.201668 28620 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac8194b8d8946d797adb804d6420646 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.202297 28607 tablet_service.cc:1511] Processing CreateTablet for tablet 841c52b2284042a8a4dab34f8ed13fe2 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.202526 28617 tablet_service.cc:1511] Processing CreateTablet for tablet 6eac9128f38545f09d093eed3c9839ae (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.202724 28614 tablet_service.cc:1511] Processing CreateTablet for tablet 392548585cf1446f99dbc268831cae8e (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.202911 28616 tablet_service.cc:1511] Processing CreateTablet for tablet 03a746a63ed14fb18d0b9f21a32a40ad (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.203096 28609 tablet_service.cc:1511] Processing CreateTablet for tablet 9fec26011dec4e91868a1cbe71cdf371 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.203270 28615 tablet_service.cc:1511] Processing CreateTablet for tablet 10733adc026c49399d8d6649c5fe7705 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.203454 28619 tablet_service.cc:1511] Processing CreateTablet for tablet acb842662fa640ee932f06fd7b5b7fc2 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.203634 28608 tablet_service.cc:1511] Processing CreateTablet for tablet b95825d687ab41878ef822421d853385 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.204010 28621 tablet_service.cc:1511] Processing CreateTablet for tablet 9b93c5f310964747a018b899b0cb2dbf (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.204834 28603 tablet_service.cc:1511] Processing CreateTablet for tablet a2d60a37346b469781732b692cf55c63 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.205034 28604 tablet_service.cc:1511] Processing CreateTablet for tablet 5745cdf610dd4552bfd339a7ed04e95c (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.205741 28606 tablet_service.cc:1511] Processing CreateTablet for tablet 437e82d85fc445ce8ae904c2c71d08c7 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.206262 28601 tablet_service.cc:1511] Processing CreateTablet for tablet b1365599d9764ccfbfb0cb954f83de69 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.206751 28602 tablet_service.cc:1511] Processing CreateTablet for tablet 6b81ec52183e4bd28ed4d17ed15f5414 (DEFAULT_TABLE table=table-0 [id=708dcc135eac410bbf10b0db8a172f84]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:43.207919 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac8194b8d8946d797adb804d6420646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.209684 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d60a37346b469781732b692cf55c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.213675 28601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1365599d9764ccfbfb0cb954f83de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.217100 28617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eac9128f38545f09d093eed3c9839ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.217617 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 392548585cf1446f99dbc268831cae8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.221693 28616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a746a63ed14fb18d0b9f21a32a40ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.225469 28609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fec26011dec4e91868a1cbe71cdf371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.226797 28615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10733adc026c49399d8d6649c5fe7705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.229656 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb842662fa640ee932f06fd7b5b7fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.233628 28606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 437e82d85fc445ce8ae904c2c71d08c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.234413 28931 tablet_bootstrap.cc:492] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.237612 28604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5745cdf610dd4552bfd339a7ed04e95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.238372 28931 tablet_bootstrap.cc:654] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.240271 28931 log.cc:826] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:43.241616 28608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b95825d687ab41878ef822421d853385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.242205 28612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd831b22d6044209063ec4b9a984e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.245582 28607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 841c52b2284042a8a4dab34f8ed13fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.246327 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b81ec52183e4bd28ed4d17ed15f5414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.255235 28621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b93c5f310964747a018b899b0cb2dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:43.259919 28930 tablet_bootstrap.cc:654] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.261916 28930 log.cc:826] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:43.272643 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.217s user 0.026s sys 0.010s Metrics: {"bytes_written":697,"cfile_cache_hit":57,"cfile_cache_hit_bytes":5701,"cfile_cache_miss":25,"cfile_cache_miss_bytes":3169,"cfile_init":5,"delete_count":0,"delta_iterators_relevant":9,"dirs.queue_time_us":194832,"dirs.run_cpu_time_us":1530,"dirs.run_wall_time_us":8260,"drs_written":1,"lbm_read_time_us":1168,"lbm_reads_lt_1ms":45,"lbm_write_time_us":1014,"lbm_writes_lt_1ms":31,"num_input_rowsets":4,"peak_mem_usage":131072,"reinsert_count":0,"rows_written":20,"spinlock_wait_cycles":14509824,"thread_start_us":21071,"threads_started":2,"update_count":16}
I20260809 07:58:43.273986 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.213476
I20260809 07:58:43.274361 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.213476
I20260809 07:58:43.274674 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.213476
I20260809 07:58:43.277496 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.277781 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.295043 28234 tablet.cc:2106] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260809 07:58:43.297397 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.013s sys 0.002s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2913,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":5388,"lbm_read_time_us":560,"lbm_reads_lt_1ms":21,"lbm_write_time_us":433,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":16}
I20260809 07:58:43.299322 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.230s user 0.029s sys 0.058s Metrics: {"dirs.queue_time_us":85033,"dirs.run_cpu_time_us":1040,"dirs.run_wall_time_us":19742,"lbm_write_time_us":423,"lbm_writes_lt_1ms":7,"mutex_wait_us":63830,"num_input_rowsets":0,"thread_start_us":131292,"threads_started":3}
I20260809 07:58:43.301218 28935 tablet_bootstrap.cc:492] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.304886 28931 tablet_bootstrap.cc:492] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.305269 28931 ts_tablet_manager.cc:1403] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.071s user 0.010s sys 0.030s
I20260809 07:58:43.306012 28930 tablet_bootstrap.cc:492] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.306339 28930 ts_tablet_manager.cc:1403] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.123s user 0.026s sys 0.021s
I20260809 07:58:43.308784 28935 tablet_bootstrap.cc:654] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.310120 28935 log.cc:826] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: Log is configured to *not* fsync() on all Append() calls
I20260809 07:58:43.312431 28935 tablet_bootstrap.cc:492] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.312700 28935 ts_tablet_manager.cc:1403] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260809 07:58:43.313277 28931 raft_consensus.cc:359] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.313735 28931 raft_consensus.cc:385] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.313846 28931 raft_consensus.cc:740] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.313695 28930 raft_consensus.cc:359] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.314085 28930 raft_consensus.cc:385] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.314194 28930 raft_consensus.cc:740] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.314563 28931 consensus_queue.cc:260] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.314803 28930 consensus_queue.cc:260] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.316890 28931 ts_tablet_manager.cc:1434] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.011s user 0.003s sys 0.006s
I20260809 07:58:43.317533 28931 tablet_bootstrap.cc:492] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.317680 28890 heartbeater.cc:499] Master 127.26.37.62:44045 was elected leader, sending a full tablet report...
I20260809 07:58:43.320365 28931 tablet_bootstrap.cc:654] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.322717 28935 raft_consensus.cc:359] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.323113 28935 raft_consensus.cc:385] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.323212 28935 raft_consensus.cc:740] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.323863 28935 consensus_queue.cc:260] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.326452 28930 ts_tablet_manager.cc:1434] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.020s user 0.009s sys 0.011s
I20260809 07:58:43.328322 28931 tablet_bootstrap.cc:492] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.328524 28931 ts_tablet_manager.cc:1403] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260809 07:58:43.328672 28930 tablet_bootstrap.cc:492] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.329929 28931 raft_consensus.cc:359] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.330127 28931 raft_consensus.cc:385] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.330237 28931 raft_consensus.cc:740] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.330598 28930 tablet_bootstrap.cc:654] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.330526 28931 consensus_queue.cc:260] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.331070 28931 ts_tablet_manager.cc:1434] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:43.331250 28935 ts_tablet_manager.cc:1434] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.018s user 0.005s sys 0.008s
I20260809 07:58:43.331444 28931 tablet_bootstrap.cc:492] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.332968 28712 heartbeater.cc:499] Master 127.26.37.62:44045 was elected leader, sending a full tablet report...
I20260809 07:58:43.333141 28935 tablet_bootstrap.cc:492] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.334824 28931 tablet_bootstrap.cc:654] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.335069 28935 tablet_bootstrap.cc:654] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.344107 28944 raft_consensus.cc:493] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.344336 28944 raft_consensus.cc:515] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.346549 28944 leader_election.cc:290] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.351022 28930 tablet_bootstrap.cc:492] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.351271 28930 ts_tablet_manager.cc:1403] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.000s
I20260809 07:58:43.352510 28930 raft_consensus.cc:359] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.352818 28930 raft_consensus.cc:385] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.352972 28930 raft_consensus.cc:740] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.353304 28930 consensus_queue.cc:260] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.353930 28930 ts_tablet_manager.cc:1434] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:43.354419 28930 tablet_bootstrap.cc:492] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.356446 28930 tablet_bootstrap.cc:654] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.367533 28935 tablet_bootstrap.cc:492] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.367738 28935 ts_tablet_manager.cc:1403] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.024s
I20260809 07:58:43.369141 28935 raft_consensus.cc:359] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.369451 28935 raft_consensus.cc:385] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.369586 28935 raft_consensus.cc:740] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.369894 28935 consensus_queue.cc:260] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.370663 28935 ts_tablet_manager.cc:1434] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:43.372830 28935 tablet_bootstrap.cc:492] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.373073 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eac9128f38545f09d093eed3c9839ae" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.375497 28935 tablet_bootstrap.cc:654] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.378623 28931 tablet_bootstrap.cc:492] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.378836 28931 ts_tablet_manager.cc:1403] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.047s user 0.017s sys 0.023s
I20260809 07:58:43.379748 28935 tablet_bootstrap.cc:492] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.379777 28931 raft_consensus.cc:359] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.379958 28935 ts_tablet_manager.cc:1403] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 07:58:43.380000 28931 raft_consensus.cc:385] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.380091 28931 raft_consensus.cc:740] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.380538 28931 consensus_queue.cc:260] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
W20260809 07:58:43.380666 28566 leader_election.cc:343] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.380810 28935 raft_consensus.cc:359] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.381031 28935 raft_consensus.cc:385] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.381130 28935 raft_consensus.cc:740] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.381624 28935 consensus_queue.cc:260] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.382369 28935 ts_tablet_manager.cc:1434] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:43.384055 28931 ts_tablet_manager.cc:1434] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260809 07:58:43.385425 28931 tablet_bootstrap.cc:492] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.387428 28931 tablet_bootstrap.cc:654] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.389791 28935 tablet_bootstrap.cc:492] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.391299 28931 tablet_bootstrap.cc:492] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.391517 28931 ts_tablet_manager.cc:1403] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 07:58:43.391779 28935 tablet_bootstrap.cc:654] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.392532 28931 raft_consensus.cc:359] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.392803 28931 raft_consensus.cc:385] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.392963 28931 raft_consensus.cc:740] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.393306 28931 consensus_queue.cc:260] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.395397 28931 ts_tablet_manager.cc:1434] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:43.395809 28931 tablet_bootstrap.cc:492] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.397207 28930 tablet_bootstrap.cc:492] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.397517 28930 ts_tablet_manager.cc:1403] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.043s user 0.017s sys 0.016s
I20260809 07:58:43.397888 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eac9128f38545f09d093eed3c9839ae" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.398518 28930 raft_consensus.cc:359] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.398794 28930 raft_consensus.cc:385] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.398927 28930 raft_consensus.cc:740] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.399202 28930 consensus_queue.cc:260] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.399869 28930 ts_tablet_manager.cc:1434] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:43.400223 28930 tablet_bootstrap.cc:492] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.401830 28935 tablet_bootstrap.cc:492] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.402004 28935 ts_tablet_manager.cc:1403] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.002s
I20260809 07:58:43.402383 28930 tablet_bootstrap.cc:654] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
W20260809 07:58:43.402758 28565 leader_election.cc:343] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.402837 28935 raft_consensus.cc:359] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.403035 28935 raft_consensus.cc:385] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.402956 28565 leader_election.cc:304] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.403134 28935 raft_consensus.cc:740] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.403368 28935 consensus_queue.cc:260] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.403935 28944 raft_consensus.cc:2749] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:43.410720 28935 ts_tablet_manager.cc:1434] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.009s user 0.000s sys 0.009s
I20260809 07:58:43.411085 28935 tablet_bootstrap.cc:492] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.411767 28930 tablet_bootstrap.cc:492] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.411976 28930 ts_tablet_manager.cc:1403] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260809 07:58:43.412833 28935 tablet_bootstrap.cc:654] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.412954 28930 raft_consensus.cc:359] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.413197 28930 raft_consensus.cc:385] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.413359 28930 raft_consensus.cc:740] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.413718 28930 consensus_queue.cc:260] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.414168 28931 tablet_bootstrap.cc:654] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.416109 28930 ts_tablet_manager.cc:1434] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:43.417228 28930 tablet_bootstrap.cc:492] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.418861 28930 tablet_bootstrap.cc:654] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.423895 28931 tablet_bootstrap.cc:492] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.424060 28931 ts_tablet_manager.cc:1403] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.028s user 0.005s sys 0.003s
I20260809 07:58:43.424140 28930 tablet_bootstrap.cc:492] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.424294 28930 ts_tablet_manager.cc:1403] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260809 07:58:43.424791 28931 raft_consensus.cc:359] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.424995 28931 raft_consensus.cc:385] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.425076 28931 raft_consensus.cc:740] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.425190 28930 raft_consensus.cc:359] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.425294 28935 tablet_bootstrap.cc:492] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.425431 28930 raft_consensus.cc:385] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.425412 28931 consensus_queue.cc:260] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.425520 28930 raft_consensus.cc:740] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.425721 28935 ts_tablet_manager.cc:1403] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260809 07:58:43.425794 28930 consensus_queue.cc:260] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.426278 28930 ts_tablet_manager.cc:1434] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:43.426579 28930 tablet_bootstrap.cc:492] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.428126 28930 tablet_bootstrap.cc:654] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.429915 28935 raft_consensus.cc:359] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.430233 28935 raft_consensus.cc:385] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.430366 28935 raft_consensus.cc:740] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.430786 28935 consensus_queue.cc:260] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.433535 28931 ts_tablet_manager.cc:1434] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260809 07:58:43.434818 28935 ts_tablet_manager.cc:1434] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260809 07:58:43.435272 28935 tablet_bootstrap.cc:492] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.436739 28930 tablet_bootstrap.cc:492] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.436874 28930 ts_tablet_manager.cc:1403] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260809 07:58:43.437237 28935 tablet_bootstrap.cc:654] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.437642 28930 raft_consensus.cc:359] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.437803 28931 tablet_bootstrap.cc:492] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.437810 28930 raft_consensus.cc:385] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.437947 28930 raft_consensus.cc:740] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.438216 28930 consensus_queue.cc:260] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.438784 28930 ts_tablet_manager.cc:1434] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:43.439157 28930 tablet_bootstrap.cc:492] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.439777 28931 tablet_bootstrap.cc:654] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.440893 28930 tablet_bootstrap.cc:654] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.444298 28935 tablet_bootstrap.cc:492] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.444479 28935 ts_tablet_manager.cc:1403] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260809 07:58:43.444664 28931 tablet_bootstrap.cc:492] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.444811 28931 ts_tablet_manager.cc:1403] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260809 07:58:43.445101 28930 tablet_bootstrap.cc:492] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.445252 28930 ts_tablet_manager.cc:1403] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:58:43.445329 28935 raft_consensus.cc:359] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.445582 28935 raft_consensus.cc:385] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.445698 28935 raft_consensus.cc:740] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.445634 28931 raft_consensus.cc:359] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.445815 28931 raft_consensus.cc:385] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.445930 28931 raft_consensus.cc:740] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.446063 28930 raft_consensus.cc:359] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.446051 28935 consensus_queue.cc:260] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.446256 28930 raft_consensus.cc:385] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.446190 28931 consensus_queue.cc:260] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.446326 28930 raft_consensus.cc:740] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.446563 28930 consensus_queue.cc:260] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.446736 28931 ts_tablet_manager.cc:1434] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:43.447077 28931 tablet_bootstrap.cc:492] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.447242 28930 ts_tablet_manager.cc:1434] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:43.448807 28931 tablet_bootstrap.cc:654] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.450569 28930 tablet_bootstrap.cc:492] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.452683 28930 tablet_bootstrap.cc:654] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.453775 28935 ts_tablet_manager.cc:1434] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260809 07:58:43.454535 28931 tablet_bootstrap.cc:492] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.454540 28935 tablet_bootstrap.cc:492] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.454679 28931 ts_tablet_manager.cc:1403] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260809 07:58:43.455468 28931 raft_consensus.cc:359] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.455674 28931 raft_consensus.cc:385] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.455754 28931 raft_consensus.cc:740] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.455984 28931 consensus_queue.cc:260] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.456317 28935 tablet_bootstrap.cc:654] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.456496 28931 ts_tablet_manager.cc:1434] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260809 07:58:43.456743 28528 tablet.cc:2406] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:58:43.457213 28930 tablet_bootstrap.cc:492] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.457388 28930 ts_tablet_manager.cc:1403] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 07:58:43.458204 28930 raft_consensus.cc:359] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.458417 28930 raft_consensus.cc:385] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.458550 28930 raft_consensus.cc:740] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.458794 28930 consensus_queue.cc:260] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.459369 28930 ts_tablet_manager.cc:1434] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:43.459667 28930 tablet_bootstrap.cc:492] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.462049 28930 tablet_bootstrap.cc:654] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.463085 28931 tablet_bootstrap.cc:492] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
W20260809 07:58:43.463835 28892 tablet.cc:2406] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:58:43.467110 28935 tablet_bootstrap.cc:492] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.467092 28942 raft_consensus.cc:493] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.467274 28935 ts_tablet_manager.cc:1403] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260809 07:58:43.467343 28942 raft_consensus.cc:515] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.468036 28935 raft_consensus.cc:359] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.468225 28935 raft_consensus.cc:385] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.468312 28935 raft_consensus.cc:740] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.468648 28935 consensus_queue.cc:260] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.469897 28942 leader_election.cc:290] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
W20260809 07:58:43.470402 28715 tablet.cc:2406] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:58:43.471390 28935 ts_tablet_manager.cc:1434] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:43.475293 28935 tablet_bootstrap.cc:492] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.477260 28935 tablet_bootstrap.cc:654] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.477674 28931 tablet_bootstrap.cc:654] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.480422 28930 tablet_bootstrap.cc:492] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.480703 28930 ts_tablet_manager.cc:1403] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.008s
I20260809 07:58:43.481820 28930 raft_consensus.cc:359] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.482185 28930 raft_consensus.cc:385] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.482352 28930 raft_consensus.cc:740] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.482807 28930 consensus_queue.cc:260] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.483433 28930 ts_tablet_manager.cc:1434] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:43.483803 28930 tablet_bootstrap.cc:492] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.486127 28930 tablet_bootstrap.cc:654] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.501699 28941 raft_consensus.cc:493] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.502029 28941 raft_consensus.cc:515] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.504223 28941 leader_election.cc:290] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.509367 28935 tablet_bootstrap.cc:492] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.509554 28935 ts_tablet_manager.cc:1403] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.034s user 0.012s sys 0.017s
I20260809 07:58:43.510641 28935 raft_consensus.cc:359] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.510869 28935 raft_consensus.cc:385] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.510969 28935 raft_consensus.cc:740] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.511276 28935 consensus_queue.cc:260] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.512025 28935 ts_tablet_manager.cc:1434] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:43.512892 28935 tablet_bootstrap.cc:492] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.515230 28935 tablet_bootstrap.cc:654] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.515491 28931 tablet_bootstrap.cc:492] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.515653 28931 ts_tablet_manager.cc:1403] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.053s user 0.010s sys 0.003s
I20260809 07:58:43.516486 28931 raft_consensus.cc:359] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.518083 28931 raft_consensus.cc:385] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.518203 28931 raft_consensus.cc:740] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.518843 28930 tablet_bootstrap.cc:492] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.518894 28931 consensus_queue.cc:260] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.519976 28930 ts_tablet_manager.cc:1403] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.036s user 0.015s sys 0.017s
I20260809 07:58:43.526697 28930 raft_consensus.cc:359] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.530154 28930 raft_consensus.cc:385] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.530342 28930 raft_consensus.cc:740] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.530714 28930 consensus_queue.cc:260] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.532548 28930 ts_tablet_manager.cc:1434] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260809 07:58:43.534114 28942 raft_consensus.cc:493] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.534328 28942 raft_consensus.cc:515] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.535050 28931 ts_tablet_manager.cc:1434] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260809 07:58:43.535135 28942 leader_election.cc:290] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.535601 28942 raft_consensus.cc:493] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.535800 28942 raft_consensus.cc:515] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.536545 28942 leader_election.cc:290] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.536811 28930 tablet_bootstrap.cc:492] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.538316 28961 raft_consensus.cc:493] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.538480 28961 raft_consensus.cc:515] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.538857 28931 tablet_bootstrap.cc:492] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.539482 28961 leader_election.cc:290] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.540140 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95825d687ab41878ef822421d853385" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.540591 28931 tablet_bootstrap.cc:654] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
W20260809 07:58:43.540877 28565 leader_election.cc:343] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.543531 28930 tablet_bootstrap.cc:654] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.543956 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b93c5f310964747a018b899b0cb2dbf" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.545012 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb842662fa640ee932f06fd7b5b7fc2" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.546888 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392548585cf1446f99dbc268831cae8e" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
W20260809 07:58:43.547869 28380 leader_election.cc:343] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:43.548308 28380 leader_election.cc:343] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:43.548738 28380 leader_election.cc:343] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.550259 28824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392548585cf1446f99dbc268831cae8e" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.551028 28824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb842662fa640ee932f06fd7b5b7fc2" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.551270 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95825d687ab41878ef822421d853385" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:43.551493 28382 leader_election.cc:343] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.551638 28382 leader_election.cc:304] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.551666 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b93c5f310964747a018b899b0cb2dbf" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.552027 28825 raft_consensus.cc:2468] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
W20260809 07:58:43.552608 28382 leader_election.cc:343] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.552651 28942 raft_consensus.cc:2749] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:43.552933 28382 leader_election.cc:304] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.553225 28942 raft_consensus.cc:2749] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 07:58:43.553673 28566 leader_election.cc:343] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.553663 28382 leader_election.cc:304] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:43.553817 28566 leader_election.cc:304] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.554056 28989 raft_consensus.cc:2804] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.554165 28989 raft_consensus.cc:493] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.554247 28961 raft_consensus.cc:2749] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:43.554311 28989 raft_consensus.cc:3060] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.557384 28989 raft_consensus.cc:515] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.558290 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fec26011dec4e91868a1cbe71cdf371" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.558547 28648 raft_consensus.cc:2468] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.559268 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fec26011dec4e91868a1cbe71cdf371" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.559494 28464 raft_consensus.cc:2468] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.560369 28740 leader_election.cc:304] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.560835 28941 raft_consensus.cc:2804] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.561012 28941 raft_consensus.cc:493] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.561143 28941 raft_consensus.cc:3060] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.563234 28935 tablet_bootstrap.cc:492] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.563300 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b93c5f310964747a018b899b0cb2dbf" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:43.563417 28935 ts_tablet_manager.cc:1403] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.051s user 0.016s sys 0.032s
I20260809 07:58:43.563468 28825 raft_consensus.cc:3060] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.564286 28935 raft_consensus.cc:359] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.564353 28941 raft_consensus.cc:515] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.564523 28935 raft_consensus.cc:385] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.564613 28935 raft_consensus.cc:740] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.565016 28930 tablet_bootstrap.cc:492] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.564913 28935 consensus_queue.cc:260] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.565013 28941 leader_election.cc:290] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.565258 28930 ts_tablet_manager.cc:1403] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.003s
I20260809 07:58:43.565673 28935 ts_tablet_manager.cc:1434] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:43.566205 28935 tablet_bootstrap.cc:492] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.566222 28930 raft_consensus.cc:359] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.566637 28930 raft_consensus.cc:385] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.566658 28825 raft_consensus.cc:2468] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:43.566810 28930 raft_consensus.cc:740] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.567181 28930 consensus_queue.cc:260] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.567339 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fec26011dec4e91868a1cbe71cdf371" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.567545 28464 raft_consensus.cc:3060] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.568177 28935 tablet_bootstrap.cc:654] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.568697 28930 ts_tablet_manager.cc:1434] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:43.569103 28930 tablet_bootstrap.cc:492] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.570751 28464 raft_consensus.cc:2468] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.571334 28961 raft_consensus.cc:493] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.571504 28961 raft_consensus.cc:515] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.571621 28930 tablet_bootstrap.cc:654] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.572207 28961 leader_election.cc:290] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.572978 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10733adc026c49399d8d6649c5fe7705" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.573275 28462 raft_consensus.cc:2468] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:43.573801 28565 leader_election.cc:304] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.574090 28961 raft_consensus.cc:2804] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.574235 28961 raft_consensus.cc:493] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.574344 28961 raft_consensus.cc:3060] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.576097 28931 tablet_bootstrap.cc:492] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.576277 28931 ts_tablet_manager.cc:1403] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.000s
I20260809 07:58:43.577015 28931 raft_consensus.cc:359] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.577203 28931 raft_consensus.cc:385] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.577272 28931 raft_consensus.cc:740] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.577747 28961 raft_consensus.cc:515] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.578168 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fec26011dec4e91868a1cbe71cdf371" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.578372 28648 raft_consensus.cc:3060] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.578491 28935 tablet_bootstrap.cc:492] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.578697 28935 ts_tablet_manager.cc:1403] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260809 07:58:43.579526 28382 leader_election.cc:304] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.579805 28942 raft_consensus.cc:2804] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.579902 28935 raft_consensus.cc:359] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.580111 28935 raft_consensus.cc:385] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.580107 28740 leader_election.cc:304] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.580224 28935 raft_consensus.cc:740] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.580547 28935 consensus_queue.cc:260] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.580900 28989 leader_election.cc:290] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.581231 28941 raft_consensus.cc:2804] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.581535 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b93c5f310964747a018b899b0cb2dbf" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.581789 28647 raft_consensus.cc:3060] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.581919 28935 ts_tablet_manager.cc:1434] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:43.582307 28935 tablet_bootstrap.cc:492] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.584780 28935 tablet_bootstrap.cc:654] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.587985 28930 tablet_bootstrap.cc:492] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.588184 28930 ts_tablet_manager.cc:1403] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.005s
I20260809 07:58:43.588111 28931 consensus_queue.cc:260] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.588948 28935 tablet_bootstrap.cc:492] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.589165 28935 ts_tablet_manager.cc:1403] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 07:58:43.589668 28930 raft_consensus.cc:359] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.590132 28930 raft_consensus.cc:385] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.590153 28935 raft_consensus.cc:359] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.590366 28935 raft_consensus.cc:385] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.590458 28935 raft_consensus.cc:740] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.590508 28930 raft_consensus.cc:740] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.590764 28935 consensus_queue.cc:260] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.591111 28930 consensus_queue.cc:260] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.591457 28935 ts_tablet_manager.cc:1434] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 07:58:43.591804 28935 tablet_bootstrap.cc:492] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.592315 28930 ts_tablet_manager.cc:1434] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 07:58:43.593560 28930 tablet_bootstrap.cc:492] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.592895 29001 raft_consensus.cc:493] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.593853 28935 tablet_bootstrap.cc:654] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.593858 29001 raft_consensus.cc:515] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.594583 28941 raft_consensus.cc:697] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:43.597632 28941 consensus_queue.cc:237] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.599306 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10733adc026c49399d8d6649c5fe7705" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.599532 28825 raft_consensus.cc:2468] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:43.599992 28647 raft_consensus.cc:2468] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:43.600191 28931 ts_tablet_manager.cc:1434] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.024s user 0.003s sys 0.002s
I20260809 07:58:43.600553 28931 tablet_bootstrap.cc:492] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.600814 28648 raft_consensus.cc:2468] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.602226 28961 leader_election.cc:290] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.603255 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10733adc026c49399d8d6649c5fe7705" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:43.603785 28935 tablet_bootstrap.cc:492] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.603958 28935 ts_tablet_manager.cc:1403] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260809 07:58:43.604949 28942 raft_consensus.cc:697] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:43.604903 28935 raft_consensus.cc:359] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.605119 28935 raft_consensus.cc:385] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.605101 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10733adc026c49399d8d6649c5fe7705" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.605213 28935 raft_consensus.cc:740] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.605316 28464 raft_consensus.cc:3060] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.605542 28935 consensus_queue.cc:260] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.605558 28942 consensus_queue.cc:237] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:43.605748 28930 tablet_bootstrap.cc:654] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.606089 28935 ts_tablet_manager.cc:1434] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260809 07:58:43.606369 28935 tablet_bootstrap.cc:492] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.609077 28825 raft_consensus.cc:3060] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.608112 28935 tablet_bootstrap.cc:654] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.613549 28931 tablet_bootstrap.cc:654] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.613689 28825 raft_consensus.cc:2468] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:43.614187 28566 leader_election.cc:304] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.614636 28961 raft_consensus.cc:2804] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.614825 28961 raft_consensus.cc:697] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:43.615473 28464 raft_consensus.cc:2468] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:43.615392 28961 consensus_queue.cc:237] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.617304 28153 catalog_manager.cc:5719] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:43.622843 29001 leader_election.cc:290] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.623687 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "841c52b2284042a8a4dab34f8ed13fe2" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.623950 28464 raft_consensus.cc:2468] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.625267 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "841c52b2284042a8a4dab34f8ed13fe2" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.625526 28647 raft_consensus.cc:2468] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.625590 28740 leader_election.cc:304] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.626066 29001 raft_consensus.cc:2804] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.626407 29001 raft_consensus.cc:493] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.626570 29001 raft_consensus.cc:3060] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.631932 28998 raft_consensus.cc:493] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.632126 28998 raft_consensus.cc:515] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.633178 28998 leader_election.cc:290] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.633919 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac8194b8d8946d797adb804d6420646" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
W20260809 07:58:43.634681 28380 leader_election.cc:343] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.635299 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac8194b8d8946d797adb804d6420646" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.635610 28825 raft_consensus.cc:2468] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:43.636073 28382 leader_election.cc:304] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:43.636950 28998 raft_consensus.cc:2804] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.637075 28998 raft_consensus.cc:493] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.637177 28998 raft_consensus.cc:3060] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.639204 28151 catalog_manager.cc:5719] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.641315 28998 raft_consensus.cc:515] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.642731 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac8194b8d8946d797adb804d6420646" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
W20260809 07:58:43.643350 28380 leader_election.cc:343] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.645812 28942 raft_consensus.cc:493] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.646011 28942 raft_consensus.cc:515] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.647195 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5745cdf610dd4552bfd339a7ed04e95c" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.647466 28647 raft_consensus.cc:2468] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:43.648031 28380 leader_election.cc:304] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.648578 28989 raft_consensus.cc:2804] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.648618 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac8194b8d8946d797adb804d6420646" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:43.648725 28989 raft_consensus.cc:493] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.648826 28989 raft_consensus.cc:3060] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.648968 28825 raft_consensus.cc:3060] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.648360 28998 leader_election.cc:290] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.652377 28825 raft_consensus.cc:2468] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:43.652809 28382 leader_election.cc:304] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:43.653192 28998 raft_consensus.cc:2804] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.652796 28989 raft_consensus.cc:515] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.654265 28989 leader_election.cc:290] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.654386 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5745cdf610dd4552bfd339a7ed04e95c" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.654577 28647 raft_consensus.cc:3060] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.655267 28942 leader_election.cc:290] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.655434 28150 catalog_manager.cc:5719] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.656919 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5745cdf610dd4552bfd339a7ed04e95c" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.657100 28826 raft_consensus.cc:2468] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:43.657783 28647 raft_consensus.cc:2468] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:43.658102 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5745cdf610dd4552bfd339a7ed04e95c" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:43.658190 28380 leader_election.cc:304] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.658416 28825 raft_consensus.cc:3060] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.658547 28942 raft_consensus.cc:2804] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.658785 28942 raft_consensus.cc:697] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:43.659086 28942 consensus_queue.cc:237] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.661710 28825 raft_consensus.cc:2468] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:43.662451 28998 raft_consensus.cc:697] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:43.662750 29001 raft_consensus.cc:515] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.662827 28998 consensus_queue.cc:237] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.663365 29001 leader_election.cc:290] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.663775 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "841c52b2284042a8a4dab34f8ed13fe2" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.663980 28464 raft_consensus.cc:3060] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.666705 28931 tablet_bootstrap.cc:492] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.666855 28931 ts_tablet_manager.cc:1403] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.066s user 0.000s sys 0.011s
I20260809 07:58:43.667079 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "841c52b2284042a8a4dab34f8ed13fe2" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.667268 28647 raft_consensus.cc:3060] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.667541 28464 raft_consensus.cc:2468] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.667744 28930 tablet_bootstrap.cc:492] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.667701 28931 raft_consensus.cc:359] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.667866 28931 raft_consensus.cc:385] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.667889 28930 ts_tablet_manager.cc:1403] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.004s
I20260809 07:58:43.667973 28931 raft_consensus.cc:740] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.668146 28740 leader_election.cc:304] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.668241 28931 consensus_queue.cc:260] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.668664 28941 raft_consensus.cc:2804] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.668687 28930 raft_consensus.cc:359] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.668833 28931 ts_tablet_manager.cc:1434] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:43.668872 28930 raft_consensus.cc:385] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.668939 28930 raft_consensus.cc:740] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.669147 28931 tablet_bootstrap.cc:492] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.669159 28930 consensus_queue.cc:260] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.669474 28941 raft_consensus.cc:697] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:43.669659 28930 ts_tablet_manager.cc:1434] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:43.669752 28941 consensus_queue.cc:237] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.669951 28930 tablet_bootstrap.cc:492] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.670001 28647 raft_consensus.cc:2468] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.671058 28931 tablet_bootstrap.cc:654] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.671751 28930 tablet_bootstrap.cc:654] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.674533 28935 tablet_bootstrap.cc:492] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.674746 28935 ts_tablet_manager.cc:1403] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.068s user 0.033s sys 0.020s
I20260809 07:58:43.675715 28935 raft_consensus.cc:359] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.675956 28935 raft_consensus.cc:385] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.676045 28935 raft_consensus.cc:740] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.676324 28935 consensus_queue.cc:260] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.677796 28935 ts_tablet_manager.cc:1434] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 07:58:43.678144 28935 tablet_bootstrap.cc:492] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.680224 28935 tablet_bootstrap.cc:654] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.683248 28935 tablet_bootstrap.cc:492] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.683470 28935 ts_tablet_manager.cc:1403] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 07:58:43.684417 28935 raft_consensus.cc:359] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.684731 28935 raft_consensus.cc:385] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.684876 28935 raft_consensus.cc:740] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.685223 28935 consensus_queue.cc:260] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.686424 28935 ts_tablet_manager.cc:1434] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:43.686882 28935 tablet_bootstrap.cc:492] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.689020 28935 tablet_bootstrap.cc:654] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.709375 28961 raft_consensus.cc:493] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.709586 28961 raft_consensus.cc:515] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.710237 29001 raft_consensus.cc:493] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.710300 28961 leader_election.cc:290] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.710444 29001 raft_consensus.cc:515] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.711267 29001 leader_election.cc:290] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.711454 29001 raft_consensus.cc:493] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.711462 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a746a63ed14fb18d0b9f21a32a40ad" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.711617 29001 raft_consensus.cc:515] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.712363 29001 leader_election.cc:290] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.712359 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b81ec52183e4bd28ed4d17ed15f5414" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.712637 28462 raft_consensus.cc:2468] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.712752 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a746a63ed14fb18d0b9f21a32a40ad" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.713034 28464 raft_consensus.cc:2468] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:43.713624 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b81ec52183e4bd28ed4d17ed15f5414" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.713850 28647 raft_consensus.cc:2468] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.713619 28565 leader_election.cc:304] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.714524 28961 raft_consensus.cc:2804] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.714636 28961 raft_consensus.cc:493] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.714781 28961 raft_consensus.cc:3060] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.718353 28961 raft_consensus.cc:515] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.719472 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a746a63ed14fb18d0b9f21a32a40ad" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.719730 28464 raft_consensus.cc:3060] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.722012 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.722357 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.722687 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
W20260809 07:58:43.723194 28566 leader_election.cc:343] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.723228 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.723407 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.724398 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.724457 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd831b22d6044209063ec4b9a984e6d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.724644 28647 raft_consensus.cc:2468] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.724884 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.725024 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd831b22d6044209063ec4b9a984e6d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.725235 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.725238 28462 raft_consensus.cc:2468] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.725589 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.725826 28464 raft_consensus.cc:2468] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:43.726219 28565 leader_election.cc:304] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.726521 28944 raft_consensus.cc:2804] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.727540 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.727834 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.728091 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.728910 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.001s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":167,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:43.729789 28740 leader_election.cc:304] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.730227 29003 raft_consensus.cc:2804] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.730371 29003 raft_consensus.cc:493] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.730499 29003 raft_consensus.cc:3060] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.730515 28961 leader_election.cc:290] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.733351 29001 raft_consensus.cc:493] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.733480 29001 raft_consensus.cc:515] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.734037 29001 leader_election.cc:290] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.734676 28961 raft_consensus.cc:493] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.734886 28961 raft_consensus.cc:515] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.736246 28961 leader_election.cc:290] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.736500 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.003s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":159,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:43.736620 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437e82d85fc445ce8ae904c2c71d08c7" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
W20260809 07:58:43.737260 28565 leader_election.cc:343] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.738489 29003 raft_consensus.cc:515] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.739322 29003 leader_election.cc:290] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.730237 28740 leader_election.cc:304] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.742164 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eac9128f38545f09d093eed3c9839ae" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.742620 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eac9128f38545f09d093eed3c9839ae" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.742844 28647 raft_consensus.cc:2468] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.743254 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b81ec52183e4bd28ed4d17ed15f5414" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.743464 28462 raft_consensus.cc:3060] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.743891 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437e82d85fc445ce8ae904c2c71d08c7" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.744637 28740 leader_election.cc:304] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.744917 29001 raft_consensus.cc:2804] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.745038 29001 raft_consensus.cc:493] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.745133 29001 raft_consensus.cc:3060] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.745250 29003 raft_consensus.cc:2804] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.745365 29003 raft_consensus.cc:493] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.745460 29003 raft_consensus.cc:3060] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.747848 28462 raft_consensus.cc:2468] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
W20260809 07:58:43.749887 28566 leader_election.cc:343] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.750043 28566 leader_election.cc:304] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.750579 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b81ec52183e4bd28ed4d17ed15f5414" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.750816 28647 raft_consensus.cc:3060] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.751967 28153 catalog_manager.cc:5719] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:43.753425 29001 raft_consensus.cc:515] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.754245 29001 leader_election.cc:290] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.754443 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a746a63ed14fb18d0b9f21a32a40ad" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:43.755721 28464 raft_consensus.cc:2468] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:43.757190 28647 raft_consensus.cc:2468] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.758112 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd831b22d6044209063ec4b9a984e6d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.758361 28464 raft_consensus.cc:3060] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.760489 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.760833 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.761124 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.761447 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.761806 28464 raft_consensus.cc:2468] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.762607 29003 raft_consensus.cc:515] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.763114 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd831b22d6044209063ec4b9a984e6d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.763201 29003 leader_election.cc:290] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:43.763338 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.763361 28647 raft_consensus.cc:3060] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.763414 28944 raft_consensus.cc:697] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:43.763607 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.763866 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.763805 28944 consensus_queue.cc:237] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.764250 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.173391
I20260809 07:58:43.764587 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.173391
I20260809 07:58:43.764891 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.173391
I20260809 07:58:43.765265 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
W20260809 07:58:43.765415 28566 leader_election.cc:343] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.765547 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.765806 29029 raft_consensus.cc:2749] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:43.765916 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:43.766345 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.767227 29029 raft_consensus.cc:493] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.767457 29029 raft_consensus.cc:515] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.768757 28647 raft_consensus.cc:2468] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.768857 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb842662fa640ee932f06fd7b5b7fc2" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.769084 28464 raft_consensus.cc:2468] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:43.769630 28565 leader_election.cc:304] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.769984 28961 raft_consensus.cc:2804] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.770103 28961 raft_consensus.cc:493] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.770256 28961 raft_consensus.cc:3060] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.771234 28740 leader_election.cc:304] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.771802 28931 tablet_bootstrap.cc:492] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.772080 28931 ts_tablet_manager.cc:1403] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.103s user 0.001s sys 0.027s
I20260809 07:58:43.772858 28931 raft_consensus.cc:359] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.773065 28931 raft_consensus.cc:385] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.773162 28931 raft_consensus.cc:740] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.773447 28931 consensus_queue.cc:260] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.774029 28931 ts_tablet_manager.cc:1434] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:43.774389 28931 tablet_bootstrap.cc:492] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.775127 28998 raft_consensus.cc:493] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.775316 28998 raft_consensus.cc:515] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.775991 28998 leader_election.cc:290] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.776257 28931 tablet_bootstrap.cc:654] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.776980 28961 raft_consensus.cc:515] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.778025 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb842662fa640ee932f06fd7b5b7fc2" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.778138 28941 raft_consensus.cc:2804] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.771801 28740 leader_election.cc:304] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:43.778225 28462 raft_consensus.cc:3060] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.778290 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.778595 29001 raft_consensus.cc:2804] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.778635 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.778743 29001 raft_consensus.cc:697] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:43.778982 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.779002 29001 consensus_queue.cc:237] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.779305 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.781558 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.781831 28961 leader_election.cc:290] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.781872 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.782225 28462 raft_consensus.cc:2468] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:43.782465 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.782761 28565 leader_election.cc:304] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.783089 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.388977
I20260809 07:58:43.783149 28961 raft_consensus.cc:2804] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.783434 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.388977
I20260809 07:58:43.783754 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.388977
I20260809 07:58:43.784260 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.784530 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.784482 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eac9128f38545f09d093eed3c9839ae" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.784658 28941 raft_consensus.cc:697] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:43.784706 28464 raft_consensus.cc:3060] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.784952 28941 consensus_queue.cc:237] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.786490 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eac9128f38545f09d093eed3c9839ae" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.786720 28648 raft_consensus.cc:3060] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.787938 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1365599d9764ccfbfb0cb954f83de69" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.789527 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb842662fa640ee932f06fd7b5b7fc2" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:43.789606 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:43.790130 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
W20260809 07:58:43.790343 28382 leader_election.cc:343] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.790654 28648 raft_consensus.cc:2468] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.791083 28740 leader_election.cc:304] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
W20260809 07:58:43.791115 28566 leader_election.cc:343] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.791507 29003 raft_consensus.cc:2804] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.791801 29003 raft_consensus.cc:697] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:43.792099 29003 consensus_queue.cc:237] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.797444 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.003s sys 0.004s Metrics: {"bytes_written":509,"delete_count":0,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260809 07:58:43.799984 28464 raft_consensus.cc:2468] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:43.802954 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1365599d9764ccfbfb0cb954f83de69" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.803241 28647 raft_consensus.cc:2468] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:43.803733 28380 leader_election.cc:304] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.804288 28998 raft_consensus.cc:2804] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.804404 28998 raft_consensus.cc:493] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.804528 28998 raft_consensus.cc:3060] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.807968 28998 raft_consensus.cc:515] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.809201 28998 leader_election.cc:290] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.809507 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1365599d9764ccfbfb0cb954f83de69" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.809710 28648 raft_consensus.cc:3060] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.812654 28648 raft_consensus.cc:2468] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:43.813095 28930 tablet_bootstrap.cc:492] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.813139 28380 leader_election.cc:304] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.813431 28930 ts_tablet_manager.cc:1403] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.144s user 0.033s sys 0.030s
I20260809 07:58:43.813645 28998 raft_consensus.cc:2804] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.813845 28998 raft_consensus.cc:697] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:43.814132 28998 consensus_queue.cc:237] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.814699 28930 raft_consensus.cc:359] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.814962 28930 raft_consensus.cc:385] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.815155 28930 raft_consensus.cc:740] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.815527 28930 consensus_queue.cc:260] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.816282 28930 ts_tablet_manager.cc:1434] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:43.816654 28930 tablet_bootstrap.cc:492] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.817875 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1365599d9764ccfbfb0cb954f83de69" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:43.818208 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.056s user 0.004s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":169,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
W20260809 07:58:43.818367 28382 leader_election.cc:343] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.820009 28930 tablet_bootstrap.cc:654] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.823768 28931 tablet_bootstrap.cc:492] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.823958 28931 ts_tablet_manager.cc:1403] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.050s user 0.008s sys 0.003s
I20260809 07:58:43.824352 28153 catalog_manager.cc:5719] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.824949 28153 catalog_manager.cc:5719] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.825631 28961 raft_consensus.cc:697] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:43.825997 28961 consensus_queue.cc:237] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.826638 29029 leader_election.cc:290] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.827410 28151 catalog_manager.cc:5719] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.828897 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.829241 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.829551 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.836243 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb842662fa640ee932f06fd7b5b7fc2" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:43.836912 28566 leader_election.cc:343] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.838603 28931 raft_consensus.cc:359] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.839095 28931 raft_consensus.cc:385] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.839778 28930 tablet_bootstrap.cc:492] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.840018 28930 ts_tablet_manager.cc:1403] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.004s
I20260809 07:58:43.840792 28930 raft_consensus.cc:359] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.841068 28930 raft_consensus.cc:385] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.841248 28930 raft_consensus.cc:740] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.841627 28930 consensus_queue.cc:260] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.843142 28930 ts_tablet_manager.cc:1434] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 07:58:43.843467 28930 tablet_bootstrap.cc:492] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:43.845199 28930 tablet_bootstrap.cc:654] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.846657 28935 tablet_bootstrap.cc:492] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.846484 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.062s user 0.016s sys 0.002s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5474,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":11700,"dirs.run_cpu_time_us":152,"dirs.run_wall_time_us":1427,"lbm_read_time_us":666,"lbm_reads_lt_1ms":27,"lbm_write_time_us":379,"lbm_writes_lt_1ms":9,"mutex_wait_us":9190,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":18577,"threads_started":1,"update_count":4}
I20260809 07:58:43.846830 28935 ts_tablet_manager.cc:1403] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.160s user 0.047s sys 0.058s
I20260809 07:58:43.847630 28935 raft_consensus.cc:359] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.847878 28935 raft_consensus.cc:385] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.847978 28935 raft_consensus.cc:740] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.848287 28935 consensus_queue.cc:260] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.848981 28935 ts_tablet_manager.cc:1434] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:43.849288 28935 tablet_bootstrap.cc:492] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:43.851538 28935 tablet_bootstrap.cc:654] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.857172 28930 tablet_bootstrap.cc:492] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:43.857366 28930 ts_tablet_manager.cc:1403] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.002s
I20260809 07:58:43.857642 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.857959 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.859472 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.859611 28149 catalog_manager.cc:5719] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:43.860459 28149 catalog_manager.cc:5719] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:43.861109 28149 catalog_manager.cc:5719] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:43.863523 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.001s sys 0.002s Metrics: {"bytes_written":481,"delete_count":0,"lbm_write_time_us":155,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260809 07:58:43.865769 28961 raft_consensus.cc:493] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.865952 28961 raft_consensus.cc:515] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.865797 28153 catalog_manager.cc:5719] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.866142 28151 catalog_manager.cc:5719] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.866658 28961 leader_election.cc:290] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.867658 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d60a37346b469781732b692cf55c63" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.867679 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d60a37346b469781732b692cf55c63" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.867853 28930 raft_consensus.cc:359] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.868112 28930 raft_consensus.cc:385] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.868230 28930 raft_consensus.cc:740] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
W20260809 07:58:43.868413 28566 leader_election.cc:343] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.868506 28930 consensus_queue.cc:260] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.868880 28464 raft_consensus.cc:2468] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:43.869851 28565 leader_election.cc:304] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.870266 29029 raft_consensus.cc:493] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.870483 28930 ts_tablet_manager.cc:1434] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.013s user 0.000s sys 0.002s
I20260809 07:58:43.870406 29029 raft_consensus.cc:515] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.871199 29029 leader_election.cc:290] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.871373 29029 raft_consensus.cc:2804] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.871505 29029 raft_consensus.cc:493] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.871611 29029 raft_consensus.cc:3060] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.873943 29029 raft_consensus.cc:515] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.874446 29029 leader_election.cc:290] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.875293 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d60a37346b469781732b692cf55c63" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.875491 28462 raft_consensus.cc:3060] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.877449 28998 raft_consensus.cc:493] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.877614 28998 raft_consensus.cc:515] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.878316 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392548585cf1446f99dbc268831cae8e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.878417 28998 leader_election.cc:290] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.878535 28464 raft_consensus.cc:2468] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:43.878696 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437e82d85fc445ce8ae904c2c71d08c7" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:43.878875 28647 raft_consensus.cc:2468] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:43.878964 28565 leader_election.cc:304] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.879206 28961 raft_consensus.cc:2804] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.879326 28961 raft_consensus.cc:493] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.879413 28961 raft_consensus.cc:3060] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.879423 28380 leader_election.cc:304] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.879361 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437e82d85fc445ce8ae904c2c71d08c7" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.879791 28998 raft_consensus.cc:2804] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:43.879930 28998 raft_consensus.cc:493] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:43.880062 28998 raft_consensus.cc:3060] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.881490 28961 raft_consensus.cc:515] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.882125 28961 leader_election.cc:290] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.882467 28462 raft_consensus.cc:2468] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:43.882876 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392548585cf1446f99dbc268831cae8e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:43.883062 28464 raft_consensus.cc:3060] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.883554 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d60a37346b469781732b692cf55c63" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
W20260809 07:58:43.884091 28382 leader_election.cc:343] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.884655 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392548585cf1446f99dbc268831cae8e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.884920 28825 raft_consensus.cc:2468] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:43.885051 28935 tablet_bootstrap.cc:492] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:43.885181 28935 ts_tablet_manager.cc:1403] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.000s
W20260809 07:58:43.885679 28566 leader_election.cc:343] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.886014 28935 raft_consensus.cc:359] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.888486 28565 leader_election.cc:304] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.891242 28961 raft_consensus.cc:2804] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.891387 28961 raft_consensus.cc:697] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:43.891688 28961 consensus_queue.cc:237] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.892212 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392548585cf1446f99dbc268831cae8e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:43.892438 28825 raft_consensus.cc:3060] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.892975 28464 raft_consensus.cc:2468] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:43.893435 28565 leader_election.cc:304] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:43.893771 29036 raft_consensus.cc:2804] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.893894 29036 raft_consensus.cc:697] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:43.894177 29036 consensus_queue.cc:237] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.894804 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.895156 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.895470 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.895751 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.886193 28935 raft_consensus.cc:385] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.895920 28935 raft_consensus.cc:740] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.896184 28935 consensus_queue.cc:260] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.897665 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.898006 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.898260 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.898716 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.730789
I20260809 07:58:43.899019 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.730789
I20260809 07:58:43.899375 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.730789
I20260809 07:58:43.899434 28825 raft_consensus.cc:2468] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:43.899731 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.899853 28935 ts_tablet_manager.cc:1434] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260809 07:58:43.899959 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.904294 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:43.904597 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.904558 28998 raft_consensus.cc:515] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.905859 28998 leader_election.cc:290] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.905961 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437e82d85fc445ce8ae904c2c71d08c7" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
W20260809 07:58:43.906824 28382 leader_election.cc:343] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:43.907142 28149 catalog_manager.cc:5719] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.907864 28149 catalog_manager.cc:5719] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.839316 28931 raft_consensus.cc:740] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.909476 28931 consensus_queue.cc:260] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.909587 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437e82d85fc445ce8ae904c2c71d08c7" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:43.909751 28647 raft_consensus.cc:3060] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.912060 28647 raft_consensus.cc:2468] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:43.912658 28380 leader_election.cc:304] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.913102 28998 raft_consensus.cc:2804] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:43.913430 28998 raft_consensus.cc:697] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:43.913631 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.004s sys 0.000s Metrics: {"bytes_written":530,"delete_count":0,"lbm_write_time_us":155,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":5}
I20260809 07:58:43.913740 28998 consensus_queue.cc:237] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.915371 28961 raft_consensus.cc:493] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:43.915604 28961 raft_consensus.cc:515] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.916862 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac8194b8d8946d797adb804d6420646" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:43.920778 28961 leader_election.cc:290] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:43.924391 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.924750 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.925122 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.926369 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac8194b8d8946d797adb804d6420646" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:43.926791 28825 raft_consensus.cc:2393] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24137fd377124538a31e92f3b6df0673 in current term 1: Already voted for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in this term.
I20260809 07:58:43.927384 28566 leader_election.cc:304] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:43.928086 28961 raft_consensus.cc:3060] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:43.932035 28961 raft_consensus.cc:2749] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:43.932972 28931 ts_tablet_manager.cc:1434] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.109s user 0.003s sys 0.000s
I20260809 07:58:43.939481 28931 tablet_bootstrap.cc:492] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.942306 28931 tablet_bootstrap.cc:654] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.942691 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":16950}
I20260809 07:58:43.943378 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":17278}
I20260809 07:58:43.943918 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.007s sys 0.000s Metrics: {"bytes_written":454,"delete_count":0,"lbm_write_time_us":181,"lbm_writes_lt_1ms":4,"mutex_wait_us":10326,"reinsert_count":0,"update_count":2}
I20260809 07:58:43.944810 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.012s sys 0.002s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5688,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":574,"dirs.run_cpu_time_us":306,"dirs.run_wall_time_us":1761,"lbm_read_time_us":469,"lbm_reads_lt_1ms":27,"lbm_write_time_us":269,"lbm_writes_lt_1ms":9,"mutex_wait_us":5541,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":8528,"threads_started":1,"update_count":8}
I20260809 07:58:43.945652 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.180s user 0.046s sys 0.063s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":6637,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":7797,"dirs.run_cpu_time_us":251,"dirs.run_wall_time_us":2458,"lbm_read_time_us":812,"lbm_reads_lt_1ms":27,"lbm_write_time_us":348,"lbm_writes_lt_1ms":9,"mutex_wait_us":46647,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":104818,"threads_started":2,"update_count":2}
I20260809 07:58:43.947449 28931 tablet_bootstrap.cc:492] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.947626 28931 ts_tablet_manager.cc:1403] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260809 07:58:43.947510 28149 catalog_manager.cc:5719] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:43.948522 28931 raft_consensus.cc:359] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.948732 28931 raft_consensus.cc:385] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.948803 28931 raft_consensus.cc:740] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.949090 28931 consensus_queue.cc:260] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.951162 28931 ts_tablet_manager.cc:1434] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:43.951493 28931 tablet_bootstrap.cc:492] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.953562 28931 tablet_bootstrap.cc:654] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.956589 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.959288 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.130s user 0.005s sys 0.000s Metrics: {"bytes_written":713,"delete_count":0,"lbm_write_time_us":211,"lbm_writes_lt_1ms":4,"mutex_wait_us":115802,"reinsert_count":0,"update_count":12}
I20260809 07:58:43.959771 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.211189
I20260809 07:58:43.964202 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.134s user 0.001s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":208,"lbm_writes_lt_1ms":4,"mutex_wait_us":129391,"reinsert_count":0,"update_count":1}
I20260809 07:58:43.964269 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.134s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":134223}
I20260809 07:58:43.964453 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":7571}
I20260809 07:58:43.965632 28931 tablet_bootstrap.cc:492] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.965847 28931 ts_tablet_manager.cc:1403] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260809 07:58:43.966073 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.966408 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.966687 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.966964 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:43.967662 28931 raft_consensus.cc:359] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.967947 28931 raft_consensus.cc:385] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:43.968097 28931 raft_consensus.cc:740] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:43.968366 28931 consensus_queue.cc:260] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:43.968537 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.968845 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.969080 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.969440 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.199934
I20260809 07:58:43.969712 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.199934
I20260809 07:58:43.969969 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.199934
I20260809 07:58:43.970327 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.970563 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.970919 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:43.971243 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:43.971961 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.002s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":136,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:43.975697 28931 ts_tablet_manager.cc:1434] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260809 07:58:43.977661 29001 consensus_queue.cc:1048] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:43.980165 28931 tablet_bootstrap.cc:492] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:43.980763 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.011s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":6988,"cfile_init":6,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":113,"lbm_read_time_us":722,"lbm_reads_lt_1ms":31,"lbm_write_time_us":318,"lbm_writes_lt_1ms":9,"mutex_wait_us":4617,"peak_mem_usage":32768,"reinsert_count":0,"update_count":13}
I20260809 07:58:43.982517 28931 tablet_bootstrap.cc:654] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:43.987833 29001 consensus_queue.cc:1048] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:43.989686 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5863,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":104,"lbm_read_time_us":567,"lbm_reads_lt_1ms":27,"lbm_write_time_us":338,"lbm_writes_lt_1ms":9,"mutex_wait_us":1673,"peak_mem_usage":32768,"reinsert_count":0,"update_count":3}
I20260809 07:58:43.999429 28931 tablet_bootstrap.cc:492] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:43.999675 28931 ts_tablet_manager.cc:1403] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.007s
I20260809 07:58:44.000557 28931 raft_consensus.cc:359] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.000816 28931 raft_consensus.cc:385] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.000926 28931 raft_consensus.cc:740] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.001253 28931 consensus_queue.cc:260] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.003216 28931 ts_tablet_manager.cc:1434] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:44.003595 28931 tablet_bootstrap.cc:492] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.007048 28931 tablet_bootstrap.cc:654] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.025883 28998 raft_consensus.cc:493] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.026129 28998 raft_consensus.cc:515] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.026871 28998 leader_election.cc:290] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:44.027637 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95825d687ab41878ef822421d853385" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.027918 28648 raft_consensus.cc:2468] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:44.028034 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95825d687ab41878ef822421d853385" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:44.028247 28825 raft_consensus.cc:2468] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:44.028661 28380 leader_election.cc:304] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:44.029289 28998 raft_consensus.cc:2804] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:44.029455 28998 raft_consensus.cc:493] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:44.029599 28998 raft_consensus.cc:3060] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.032629 28998 raft_consensus.cc:515] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.033777 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95825d687ab41878ef822421d853385" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:44.034017 28648 raft_consensus.cc:3060] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.036957 28648 raft_consensus.cc:2468] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:44.037079 28931 tablet_bootstrap.cc:492] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.037225 28931 ts_tablet_manager.cc:1403] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.021s
I20260809 07:58:44.037636 28998 leader_election.cc:290] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:44.038069 28931 raft_consensus.cc:359] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.038164 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95825d687ab41878ef822421d853385" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:44.038256 28931 raft_consensus.cc:385] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.038339 28931 raft_consensus.cc:740] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.038388 28825 raft_consensus.cc:3060] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.038570 28931 consensus_queue.cc:260] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.041144 28825 raft_consensus.cc:2468] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:44.041596 28931 ts_tablet_manager.cc:1434] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 07:58:44.042305 28380 leader_election.cc:304] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:44.043927 28998 raft_consensus.cc:2804] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:44.044150 28998 raft_consensus.cc:697] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:44.044463 28998 consensus_queue.cc:237] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.047719 28153 catalog_manager.cc:5719] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:44.050051 28941 raft_consensus.cc:493] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.050266 28941 raft_consensus.cc:515] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.051023 28941 leader_election.cc:290] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.051656 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1365599d9764ccfbfb0cb954f83de69" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.051819 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1365599d9764ccfbfb0cb954f83de69" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.052120 28648 raft_consensus.cc:2393] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a091b20dbc4ed2b3bf6c960df964c5 in current term 1: Already voted for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in this term.
I20260809 07:58:44.052745 28740 leader_election.cc:304] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:44.053105 28941 raft_consensus.cc:3060] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.056248 28941 raft_consensus.cc:2749] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:44.062579 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.062950 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.063279 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.064103 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.064440 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.064859 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.065210 28998 consensus_queue.cc:1048] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.067776 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":198,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.069137 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":149,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.071821 28942 consensus_queue.cc:1048] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.087664 28941 raft_consensus.cc:493] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.087819 28941 raft_consensus.cc:515] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.088449 28941 leader_election.cc:290] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.088984 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a746a63ed14fb18d0b9f21a32a40ad" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.089062 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a746a63ed14fb18d0b9f21a32a40ad" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.089295 28464 raft_consensus.cc:2393] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a091b20dbc4ed2b3bf6c960df964c5 in current term 1: Already voted for candidate 24137fd377124538a31e92f3b6df0673 in this term.
I20260809 07:58:44.089808 28740 leader_election.cc:304] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:44.090101 28941 raft_consensus.cc:3060] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.091987 28941 raft_consensus.cc:2749] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:44.104235 28998 consensus_queue.cc:1048] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.109678 29003 consensus_queue.cc:1048] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.111123 28998 consensus_queue.cc:1048] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.118597 29001 consensus_queue.cc:1048] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.121394 28998 consensus_queue.cc:1048] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.133692 28989 consensus_queue.cc:1048] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.134232 28961 consensus_queue.cc:1048] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.137692 29003 raft_consensus.cc:493] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.137887 29003 raft_consensus.cc:515] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.138561 29003 leader_election.cc:290] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.139916 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb842662fa640ee932f06fd7b5b7fc2" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.141093 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb842662fa640ee932f06fd7b5b7fc2" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.141435 28464 raft_consensus.cc:2393] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a091b20dbc4ed2b3bf6c960df964c5 in current term 1: Already voted for candidate 24137fd377124538a31e92f3b6df0673 in this term.
I20260809 07:58:44.143049 28740 leader_election.cc:304] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:44.143476 29003 raft_consensus.cc:3060] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.143574 28944 consensus_queue.cc:1048] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.146808 29003 raft_consensus.cc:2749] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:44.169734 29001 consensus_queue.cc:1048] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.175148 29003 consensus_queue.cc:1048] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.206159 29003 consensus_queue.cc:1048] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.212570 29001 consensus_queue.cc:1048] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.220821 28153 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51156:
table { table_name: "table-0" } modify_external_catalogs: true
I20260809 07:58:44.221179 28153 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51156:
table { table_name: "table-0" } modify_external_catalogs: true
I20260809 07:58:44.231323 28944 consensus_queue.cc:1048] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.233539 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.233960 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.234311 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.234686 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.235119 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.235258 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 9b93c5f310964747a018b899b0cb2dbf
I20260809 07:58:44.235437 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.235709 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.235981 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.236364 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.236460 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 8ac8194b8d8946d797adb804d6420646
I20260809 07:58:44.236725 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.237028 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.237303 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.237527 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet acb842662fa640ee932f06fd7b5b7fc2
I20260809 07:58:44.237699 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.238029 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.237972 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.238281 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.238449 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.238511 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.238720 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.238745 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 6eac9128f38545f09d093eed3c9839ae
I20260809 07:58:44.239077 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b93c5f310964747a018b899b0cb2dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.239197 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.239718 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.239878 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ac8194b8d8946d797adb804d6420646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.240022 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 03a746a63ed14fb18d0b9f21a32a40ad
I20260809 07:58:44.240103 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet acb842662fa640ee932f06fd7b5b7fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.240350 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eac9128f38545f09d093eed3c9839ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.240667 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 10733adc026c49399d8d6649c5fe7705
I20260809 07:58:44.240839 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 03a746a63ed14fb18d0b9f21a32a40ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.241233 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 10733adc026c49399d8d6649c5fe7705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.241477 28782 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b93c5f310964747a018b899b0cb2dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.241642 28784 tablet_service.cc:1558] Processing DeleteTablet for tablet acb842662fa640ee932f06fd7b5b7fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.241765 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.002s sys 0.003s Metrics: {"bytes_written":913,"delete_count":0,"lbm_write_time_us":183,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260809 07:58:44.241880 28786 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ac8194b8d8946d797adb804d6420646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.242169 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet 03a746a63ed14fb18d0b9f21a32a40ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.242117 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eac9128f38545f09d093eed3c9839ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.242669 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 392548585cf1446f99dbc268831cae8e
I20260809 07:58:44.243049 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 10733adc026c49399d8d6649c5fe7705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.243227 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 9bd831b22d6044209063ec4b9a984e6d
I20260809 07:58:44.243255 28620 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b93c5f310964747a018b899b0cb2dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.243772 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 9fec26011dec4e91868a1cbe71cdf371
I20260809 07:58:44.243846 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ac8194b8d8946d797adb804d6420646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.244117 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet acb842662fa640ee932f06fd7b5b7fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.244271 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet b95825d687ab41878ef822421d853385
I20260809 07:58:44.244299 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bd831b22d6044209063ec4b9a984e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.244345 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eac9128f38545f09d093eed3c9839ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.244563 28619 tablet_service.cc:1558] Processing DeleteTablet for tablet 03a746a63ed14fb18d0b9f21a32a40ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.244666 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 392548585cf1446f99dbc268831cae8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.244776 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 841c52b2284042a8a4dab34f8ed13fe2
I20260809 07:58:44.244782 28607 tablet_service.cc:1558] Processing DeleteTablet for tablet 10733adc026c49399d8d6649c5fe7705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.245287 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 437e82d85fc445ce8ae904c2c71d08c7
I20260809 07:58:44.245878 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 5745cdf610dd4552bfd339a7ed04e95c
I20260809 07:58:44.246344 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet a2d60a37346b469781732b692cf55c63
I20260809 07:58:44.246323 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.246366 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet b95825d687ab41878ef822421d853385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.246639 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.246843 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 6b81ec52183e4bd28ed4d17ed15f5414
I20260809 07:58:44.246925 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.247047 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fec26011dec4e91868a1cbe71cdf371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.247195 28435 tablet_service.cc:1558] Processing DeleteTablet for tablet 437e82d85fc445ce8ae904c2c71d08c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.247340 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet b1365599d9764ccfbfb0cb954f83de69
I20260809 07:58:44.247786 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.248102 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.248370 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 841c52b2284042a8a4dab34f8ed13fe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.248517 28791 tablet_service.cc:1558] Processing DeleteTablet for tablet 437e82d85fc445ce8ae904c2c71d08c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.248673 28786 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fec26011dec4e91868a1cbe71cdf371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.248708 28798 tablet_service.cc:1558] Processing DeleteTablet for tablet 5745cdf610dd4552bfd339a7ed04e95c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.248775 28795 tablet_service.cc:1558] Processing DeleteTablet for tablet 841c52b2284042a8a4dab34f8ed13fe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.248961 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet a2d60a37346b469781732b692cf55c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.248989 28794 tablet_service.cc:1558] Processing DeleteTablet for tablet b95825d687ab41878ef822421d853385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.249127 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bd831b22d6044209063ec4b9a984e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.249308 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 392548585cf1446f99dbc268831cae8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.251317 28616 tablet_service.cc:1558] Processing DeleteTablet for tablet 437e82d85fc445ce8ae904c2c71d08c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.251694 28435 tablet_service.cc:1558] Processing DeleteTablet for tablet a2d60a37346b469781732b692cf55c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.251868 28616 tablet_service.cc:1558] Processing DeleteTablet for tablet 5745cdf610dd4552bfd339a7ed04e95c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.251924 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 5745cdf610dd4552bfd339a7ed04e95c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.252125 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet b1365599d9764ccfbfb0cb954f83de69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.252120 28617 tablet_service.cc:1558] Processing DeleteTablet for tablet a2d60a37346b469781732b692cf55c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.252398 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b81ec52183e4bd28ed4d17ed15f5414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:44.252468 28617 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b81ec52183e4bd28ed4d17ed15f5414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.252815 28617 tablet_service.cc:1558] Processing DeleteTablet for tablet b1365599d9764ccfbfb0cb954f83de69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.253084 28784 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b81ec52183e4bd28ed4d17ed15f5414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.253304 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet b1365599d9764ccfbfb0cb954f83de69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:44.255285 29001 consensus_queue.cc:1048] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.257534 28607 tablet_service.cc:1558] Processing DeleteTablet for tablet 392548585cf1446f99dbc268831cae8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.257830 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fec26011dec4e91868a1cbe71cdf371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.258105 28619 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bd831b22d6044209063ec4b9a984e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.258211 28153 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51156:
name: "table-1"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
I20260809 07:58:44.258342 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet 841c52b2284042a8a4dab34f8ed13fe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:44.258558 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet b95825d687ab41878ef822421d853385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:44 UTC) from {username='slave'} at 127.0.0.1:40438
W20260809 07:58:44.258970 28153 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:58:44.261052 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.261539 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.262706 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.003s sys 0.003s Metrics: {"bytes_written":913,"delete_count":0,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260809 07:58:44.265472 28944 consensus_queue.cc:1048] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.280988 29086 tablet_replica.cc:333] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.281725 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.281744 28989 consensus_queue.cc:1048] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.284958 28961 consensus_queue.cc:1048] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.286346 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.013s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":7669,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":302,"lbm_read_time_us":624,"lbm_reads_lt_1ms":27,"lbm_write_time_us":415,"lbm_writes_lt_1ms":9,"mutex_wait_us":5711,"peak_mem_usage":32768,"reinsert_count":0,"update_count":18}
I20260809 07:58:44.287137 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.296782 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.059s user 0.009s sys 0.004s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":6433,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":2796,"dirs.run_cpu_time_us":119,"dirs.run_wall_time_us":705,"lbm_read_time_us":726,"lbm_reads_lt_1ms":27,"lbm_write_time_us":385,"lbm_writes_lt_1ms":9,"mutex_wait_us":6206,"peak_mem_usage":32768,"reinsert_count":0,"update_count":18}
I20260809 07:58:44.298532 29086 raft_consensus.cc:2243] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.300184 29086 raft_consensus.cc:2272] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.301430 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.010s sys 0.003s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":268,"dirs.run_cpu_time_us":90,"dirs.run_wall_time_us":670,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":888,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260809 07:58:44.302286 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:44.302603 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:44.302942 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.345091
I20260809 07:58:44.303268 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.303540 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.345091
I20260809 07:58:44.313475 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":308,"lbm_reads_lt_1ms":4}
I20260809 07:58:44.314087 29086 ts_tablet_manager.cc:1916] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.324424 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.324775 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.325224 28941 consensus_queue.cc:1048] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.326591 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.044s user 0.016s sys 0.000s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":179,"dirs.run_wall_time_us":1139,"drs_written":1,"lbm_read_time_us":116,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1247,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260809 07:58:44.328470 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:44.329757 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:44.330132 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.330849 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.332726 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":3209,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260809 07:58:44.333194 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.334376 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.334828 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.336732 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.337813 29085 tablet_replica.cc:333] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:44.338430 29085 raft_consensus.cc:2243] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.339915 28234 tablet.cc:2106] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260809 07:58:44.343011 29086 ts_tablet_manager.cc:1929] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.343178 29086 log.cc:1199] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/9b93c5f310964747a018b899b0cb2dbf
I20260809 07:58:44.343832 29086 ts_tablet_manager.cc:1950] T 9b93c5f310964747a018b899b0cb2dbf P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.346012 29086 tablet_replica.cc:333] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.346230 29086 raft_consensus.cc:2243] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.346410 29086 raft_consensus.cc:2272] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.347342 29086 ts_tablet_manager.cc:1916] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.349128 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.018s sys 0.019s Metrics: {"dirs.queue_time_us":2361,"dirs.run_cpu_time_us":921,"dirs.run_wall_time_us":6670,"lbm_write_time_us":483,"lbm_writes_lt_1ms":7,"mutex_wait_us":4882,"num_input_rowsets":0,"spinlock_wait_cycles":12160,"thread_start_us":30846,"threads_started":1}
I20260809 07:58:44.349637 28432 tablet_service.cc:1511] Processing CreateTablet for tablet 6265b1b590cc40739365212572bfc6ed (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.350025 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6265b1b590cc40739365212572bfc6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.350507 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.009s sys 0.001s Metrics: {"bytes_written":683,"delete_count":0,"lbm_write_time_us":258,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":16}
I20260809 07:58:44.352941 29085 raft_consensus.cc:2272] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.353607 29086 ts_tablet_manager.cc:1929] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.353732 29086 log.cc:1199] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/8ac8194b8d8946d797adb804d6420646
I20260809 07:58:44.354110 29087 tablet_replica.cc:333] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.354154 29086 ts_tablet_manager.cc:1950] T 8ac8194b8d8946d797adb804d6420646 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.354153 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.354557 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.354684 29087 raft_consensus.cc:2243] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.355042 29087 raft_consensus.cc:2272] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.355199 29086 tablet_replica.cc:333] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.355170 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 75642018d47b45e6ba74466c6d5ee95e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.355361 29086 raft_consensus.cc:2243] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.355543 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75642018d47b45e6ba74466c6d5ee95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.355563 29086 raft_consensus.cc:2272] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.356138 29086 ts_tablet_manager.cc:1916] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.358716 29087 ts_tablet_manager.cc:1916] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.362196 29014 consensus_queue.cc:1048] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.365937 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 9b93c5f310964747a018b899b0cb2dbf (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.366660 28782 tablet_service.cc:1511] Processing CreateTablet for tablet 6265b1b590cc40739365212572bfc6ed (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.366920 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.367143 28784 tablet_service.cc:1511] Processing CreateTablet for tablet 75642018d47b45e6ba74466c6d5ee95e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.367733 28782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6265b1b590cc40739365212572bfc6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.367910 29086 ts_tablet_manager.cc:1929] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 07:58:44.368054 29086 log.cc:1199] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/03a746a63ed14fb18d0b9f21a32a40ad
I20260809 07:58:44.368213 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.368522 29086 ts_tablet_manager.cc:1950] T 03a746a63ed14fb18d0b9f21a32a40ad P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.369103 28784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75642018d47b45e6ba74466c6d5ee95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.369553 29086 tablet_replica.cc:333] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.369757 29086 raft_consensus.cc:2243] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.370298 29086 raft_consensus.cc:2272] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.371114 29086 ts_tablet_manager.cc:1916] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.376277 29087 ts_tablet_manager.cc:1929] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.376456 29087 log.cc:1199] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/9b93c5f310964747a018b899b0cb2dbf
I20260809 07:58:44.377092 29087 ts_tablet_manager.cc:1950] T 9b93c5f310964747a018b899b0cb2dbf P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.367565 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 8ac8194b8d8946d797adb804d6420646 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.377615 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 03a746a63ed14fb18d0b9f21a32a40ad (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.378746 29087 tablet_replica.cc:333] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.379580 29087 raft_consensus.cc:2243] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.379747 29087 raft_consensus.cc:2272] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.380555 29087 ts_tablet_manager.cc:1916] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.381515 28931 tablet_bootstrap.cc:492] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.382851 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":47153}
I20260809 07:58:44.383085 29086 ts_tablet_manager.cc:1929] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.383227 29086 log.cc:1199] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/6eac9128f38545f09d093eed3c9839ae
I20260809 07:58:44.383442 28931 tablet_bootstrap.cc:654] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.383420 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.894986
I20260809 07:58:44.383757 29086 ts_tablet_manager.cc:1950] T 6eac9128f38545f09d093eed3c9839ae P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
W20260809 07:58:44.385282 28566 consensus_peers.cc:597] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03a746a63ed14fb18d0b9f21a32a40ad. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:44.389663 29095 tablet_bootstrap.cc:492] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.391341 29095 tablet_bootstrap.cc:654] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.391268 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.056s user 0.005s sys 0.001s Metrics: {"bytes_written":614,"delete_count":0,"lbm_write_time_us":158,"lbm_writes_lt_1ms":4,"mutex_wait_us":12899,"reinsert_count":0,"update_count":12}
I20260809 07:58:44.392099 29086 tablet_replica.cc:333] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.392331 29086 raft_consensus.cc:2243] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.392570 29086 raft_consensus.cc:2272] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.393304 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 3c44aace20a640ef8aca975a73af0d93 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.393519 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.063s user 0.001s sys 0.004s Metrics: {"bytes_written":475,"delete_count":0,"lbm_write_time_us":181,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":4}
I20260809 07:58:44.393929 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c44aace20a640ef8aca975a73af0d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.394194 28437 tablet_service.cc:1511] Processing CreateTablet for tablet 33198e831822470db98234b54d09c6bc (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.394402 29086 ts_tablet_manager.cc:1916] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.394603 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33198e831822470db98234b54d09c6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.397684 28428 tablet_service.cc:1511] Processing CreateTablet for tablet f47e81a4dd5943f7873144e373bab017 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.397871 28435 tablet_service.cc:1511] Processing CreateTablet for tablet 39664b79fe3246dfbae1cb687bfc8129 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.398058 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f47e81a4dd5943f7873144e373bab017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.398268 28435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39664b79fe3246dfbae1cb687bfc8129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.398614 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 193f851a4c1043e1af5aaf4b7190730d (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.398917 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 193f851a4c1043e1af5aaf4b7190730d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.403270 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.066s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":65935}
I20260809 07:58:44.410003 29003 raft_consensus.cc:493] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.410152 29003 raft_consensus.cc:515] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.410765 29003 leader_election.cc:290] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.412045 29087 ts_tablet_manager.cc:1929] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.412070 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d60a37346b469781732b692cf55c63" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.412210 29087 log.cc:1199] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/8ac8194b8d8946d797adb804d6420646
I20260809 07:58:44.412295 28462 raft_consensus.cc:2393] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a091b20dbc4ed2b3bf6c960df964c5 in current term 1: Already voted for candidate 24137fd377124538a31e92f3b6df0673 in this term.
I20260809 07:58:44.412693 29087 ts_tablet_manager.cc:1950] T 8ac8194b8d8946d797adb804d6420646 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.413863 29087 tablet_replica.cc:333] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.413935 28931 tablet_bootstrap.cc:492] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.414099 28931 ts_tablet_manager.cc:1403] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.033s user 0.005s sys 0.002s
I20260809 07:58:44.414086 29087 raft_consensus.cc:2243] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.414572 29087 raft_consensus.cc:2272] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.414781 28931 raft_consensus.cc:359] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.414965 28931 raft_consensus.cc:385] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.415042 28931 raft_consensus.cc:740] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.415297 28931 consensus_queue.cc:260] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.415438 29087 ts_tablet_manager.cc:1916] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.415797 28931 ts_tablet_manager.cc:1434] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:44.416111 28931 tablet_bootstrap.cc:492] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.418556 28931 tablet_bootstrap.cc:654] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.418843 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d60a37346b469781732b692cf55c63" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.419931 28740 leader_election.cc:304] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:44.422441 29087 ts_tablet_manager.cc:1929] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.422566 29087 log.cc:1199] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/acb842662fa640ee932f06fd7b5b7fc2
I20260809 07:58:44.422959 29087 ts_tablet_manager.cc:1950] T acb842662fa640ee932f06fd7b5b7fc2 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.423806 29087 tablet_replica.cc:333] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.424045 29087 raft_consensus.cc:2243] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.424213 29087 raft_consensus.cc:2272] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.425025 29087 ts_tablet_manager.cc:1916] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.427975 29014 consensus_queue.cc:1048] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.428584 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1603 bytes on disk
I20260809 07:58:44.428901 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1603 bytes on disk
I20260809 07:58:44.428907 29003 raft_consensus.cc:3060] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.429169 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1603 bytes on disk
I20260809 07:58:44.429802 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4}
I20260809 07:58:44.430226 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.430500 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.432087 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 6eac9128f38545f09d093eed3c9839ae (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.434176 29087 ts_tablet_manager.cc:1929] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.434332 29087 log.cc:1199] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/6eac9128f38545f09d093eed3c9839ae
I20260809 07:58:44.434818 29087 ts_tablet_manager.cc:1950] T 6eac9128f38545f09d093eed3c9839ae P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.435787 29087 tablet_replica.cc:333] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.436010 29087 raft_consensus.cc:2243] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.436313 29087 pending_rounds.cc:70] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Trying to abort 1 pending ops.
I20260809 07:58:44.436566 29087 pending_rounds.cc:77] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316530478128066560 op_type: NO_OP noop_request { }
I20260809 07:58:44.436758 29087 raft_consensus.cc:2889] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 07:58:44.436884 29087 raft_consensus.cc:2272] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.437796 28784 tablet_service.cc:1511] Processing CreateTablet for tablet 3c44aace20a640ef8aca975a73af0d93 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.438005 29087 ts_tablet_manager.cc:1916] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.438253 28784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c44aace20a640ef8aca975a73af0d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.439703 28794 tablet_service.cc:1511] Processing CreateTablet for tablet f47e81a4dd5943f7873144e373bab017 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.440271 28794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f47e81a4dd5943f7873144e373bab017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.440639 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 193f851a4c1043e1af5aaf4b7190730d (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.444439 29087 ts_tablet_manager.cc:1929] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.444586 29087 log.cc:1199] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/03a746a63ed14fb18d0b9f21a32a40ad
I20260809 07:58:44.445019 29087 ts_tablet_manager.cc:1950] T 03a746a63ed14fb18d0b9f21a32a40ad P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.446005 29087 tablet_replica.cc:333] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.446199 29087 raft_consensus.cc:2243] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.446560 29087 raft_consensus.cc:2272] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.447417 29087 ts_tablet_manager.cc:1916] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.447531 28782 tablet_service.cc:1511] Processing CreateTablet for tablet 39664b79fe3246dfbae1cb687bfc8129 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.449246 29003 raft_consensus.cc:2749] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:44.450794 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.067s user 0.033s sys 0.000s Metrics: {"bytes_written":18497,"cfile_cache_hit":25,"cfile_cache_hit_bytes":9477,"cfile_cache_miss":63,"cfile_cache_miss_bytes":17079,"cfile_init":17,"delta_iterators_relevant":20,"dirs.queue_time_us":54,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":1159,"drs_written":1,"lbm_read_time_us":3225,"lbm_reads_lt_1ms":131,"lbm_write_time_us":971,"lbm_writes_lt_1ms":27,"num_input_rowsets":3,"peak_mem_usage":98304,"rows_written":37}
I20260809 07:58:44.451560 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1844 bytes on disk
I20260809 07:58:44.451833 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1844 bytes on disk
I20260809 07:58:44.452085 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1844 bytes on disk
I20260809 07:58:44.452304 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1844 bytes on disk
I20260809 07:58:44.452860 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":69,"lbm_reads_lt_1ms":4}
I20260809 07:58:44.453255 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.453584 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.453788 29087 ts_tablet_manager.cc:1929] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.453905 29087 log.cc:1199] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/10733adc026c49399d8d6649c5fe7705
I20260809 07:58:44.453869 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.454362 29087 ts_tablet_manager.cc:1950] T 10733adc026c49399d8d6649c5fe7705 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.455294 29087 tablet_replica.cc:333] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.455956 28620 tablet_service.cc:1511] Processing CreateTablet for tablet 6265b1b590cc40739365212572bfc6ed (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.456218 28602 tablet_service.cc:1511] Processing CreateTablet for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.456382 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6265b1b590cc40739365212572bfc6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.456662 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.457090 28617 tablet_service.cc:1511] Processing CreateTablet for tablet f47e81a4dd5943f7873144e373bab017 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.457552 28617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f47e81a4dd5943f7873144e373bab017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.457775 28607 tablet_service.cc:1511] Processing CreateTablet for tablet 33198e831822470db98234b54d09c6bc (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.458181 28607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33198e831822470db98234b54d09c6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.458649 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 9b93c5f310964747a018b899b0cb2dbf (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.458877 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 8ac8194b8d8946d797adb804d6420646 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.459053 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet acb842662fa640ee932f06fd7b5b7fc2 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.459216 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 6eac9128f38545f09d093eed3c9839ae (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.459369 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 03a746a63ed14fb18d0b9f21a32a40ad (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.459528 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 10733adc026c49399d8d6649c5fe7705 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.461719 28616 tablet_service.cc:1511] Processing CreateTablet for tablet 193f851a4c1043e1af5aaf4b7190730d (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.461755 28603 tablet_service.cc:1511] Processing CreateTablet for tablet 39664b79fe3246dfbae1cb687bfc8129 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.462065 28616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 193f851a4c1043e1af5aaf4b7190730d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.462520 28619 tablet_service.cc:1511] Processing CreateTablet for tablet 3c44aace20a640ef8aca975a73af0d93 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.462893 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c44aace20a640ef8aca975a73af0d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.465519 29029 consensus_queue.cc:1048] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.465668 28614 tablet_service.cc:1511] Processing CreateTablet for tablet 75642018d47b45e6ba74466c6d5ee95e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.466065 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75642018d47b45e6ba74466c6d5ee95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.469496 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39664b79fe3246dfbae1cb687bfc8129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.478365 28795 tablet_service.cc:1511] Processing CreateTablet for tablet 2a54a36e22474698a198d8c25bafa16b (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.480597 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 12a98f94e5ed406ba4ac318dc3699913 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.481671 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a98f94e5ed406ba4ac318dc3699913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.485006 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 2a54a36e22474698a198d8c25bafa16b (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.486258 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a54a36e22474698a198d8c25bafa16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.486501 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 193f851a4c1043e1af5aaf4b7190730d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.486799 28782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39664b79fe3246dfbae1cb687bfc8129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.486900 28797 tablet_service.cc:1511] Processing CreateTablet for tablet 33198e831822470db98234b54d09c6bc (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.487025 28795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a54a36e22474698a198d8c25bafa16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.487250 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33198e831822470db98234b54d09c6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.486249 29086 ts_tablet_manager.cc:1929] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.493804 29086 log.cc:1199] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/10733adc026c49399d8d6649c5fe7705
I20260809 07:58:44.494282 29086 ts_tablet_manager.cc:1950] T 10733adc026c49399d8d6649c5fe7705 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.495642 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 10733adc026c49399d8d6649c5fe7705 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.496697 29014 consensus_queue.cc:1048] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.505304 29003 raft_consensus.cc:493] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.505627 29003 raft_consensus.cc:515] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.509007 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437e82d85fc445ce8ae904c2c71d08c7" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.509917 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437e82d85fc445ce8ae904c2c71d08c7" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.514377 29003 leader_election.cc:290] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.516258 29086 tablet_replica.cc:333] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.516384 29029 consensus_queue.cc:1048] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.517519 29029 consensus_queue.cc:1048] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.517670 28614 tablet_service.cc:1511] Processing CreateTablet for tablet 2a54a36e22474698a198d8c25bafa16b (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.518121 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a54a36e22474698a198d8c25bafa16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.519059 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 12a98f94e5ed406ba4ac318dc3699913 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.519716 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a98f94e5ed406ba4ac318dc3699913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.518446 29029 consensus_queue.cc:1048] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.523613 29086 raft_consensus.cc:2243] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.523877 29086 raft_consensus.cc:2272] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.524487 28740 leader_election.cc:304] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
W20260809 07:58:44.524775 28740 raft_consensus.cc:2686] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 07:58:44.525538 28382 consensus_peers.cc:597] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:44.518996 28603 tablet_service.cc:1511] Processing CreateTablet for tablet 12a98f94e5ed406ba4ac318dc3699913 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.526793 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a98f94e5ed406ba4ac318dc3699913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.534765 29086 ts_tablet_manager.cc:1916] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.538518 28432 tablet_service.cc:1511] Processing CreateTablet for tablet 7751d1e95a124184848a095dfcc6268e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.539028 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7751d1e95a124184848a095dfcc6268e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.544337 28797 tablet_service.cc:1511] Processing CreateTablet for tablet 7751d1e95a124184848a095dfcc6268e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.544929 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7751d1e95a124184848a095dfcc6268e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.556558 29086 ts_tablet_manager.cc:1929] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 07:58:44.556740 29086 log.cc:1199] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/437e82d85fc445ce8ae904c2c71d08c7
I20260809 07:58:44.557183 29086 ts_tablet_manager.cc:1950] T 437e82d85fc445ce8ae904c2c71d08c7 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.558465 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 437e82d85fc445ce8ae904c2c71d08c7 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.561978 28989 consensus_queue.cc:1048] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:44.562206 29086 tablet_replica.cc:333] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.562424 29086 raft_consensus.cc:2243] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.562789 29086 raft_consensus.cc:2272] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.569844 28931 tablet_bootstrap.cc:492] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.570010 28931 ts_tablet_manager.cc:1403] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.154s user 0.016s sys 0.018s
I20260809 07:58:44.570804 28931 raft_consensus.cc:359] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.571003 28931 raft_consensus.cc:385] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.571106 28931 raft_consensus.cc:740] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.571318 28931 consensus_queue.cc:260] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.571851 28931 ts_tablet_manager.cc:1434] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:44.572171 28931 tablet_bootstrap.cc:492] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.574096 28931 tablet_bootstrap.cc:654] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.575346 28619 tablet_service.cc:1511] Processing CreateTablet for tablet 7751d1e95a124184848a095dfcc6268e (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.575754 28428 tablet_service.cc:1511] Processing CreateTablet for tablet fc1ec621e45e4e57b7fb536f3db739eb (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.576303 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1ec621e45e4e57b7fb536f3db739eb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 07:58:44.581103 28382 consensus_peers.cc:597] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ac8194b8d8946d797adb804d6420646. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:44.582419 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7751d1e95a124184848a095dfcc6268e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.582551 29086 ts_tablet_manager.cc:1916] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.586454 29095 tablet_bootstrap.cc:492] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:44.587096 28931 tablet_bootstrap.cc:492] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.587307 29095 ts_tablet_manager.cc:1403] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.198s user 0.018s sys 0.019s
I20260809 07:58:44.588174 29095 raft_consensus.cc:359] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.588531 28931 ts_tablet_manager.cc:1403] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20260809 07:58:44.589368 28931 raft_consensus.cc:359] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.589610 29095 raft_consensus.cc:385] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.589758 29095 raft_consensus.cc:740] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.589952 28931 raft_consensus.cc:385] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.591005 29095 consensus_queue.cc:260] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.591260 28931 raft_consensus.cc:740] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.591643 28931 consensus_queue.cc:260] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.592347 29095 ts_tablet_manager.cc:1434] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260809 07:58:44.593111 29095 tablet_bootstrap.cc:492] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.595148 28931 ts_tablet_manager.cc:1434] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260809 07:58:44.597205 29095 tablet_bootstrap.cc:654] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.597601 28931 tablet_bootstrap.cc:492] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.599763 28931 tablet_bootstrap.cc:654] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.600492 29086 ts_tablet_manager.cc:1929] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.605132 29087 raft_consensus.cc:2243] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.605178 29086 log.cc:1199] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/9fec26011dec4e91868a1cbe71cdf371
I20260809 07:58:44.606001 29087 pending_rounds.cc:70] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Trying to abort 1 pending ops.
I20260809 07:58:44.605782 29086 ts_tablet_manager.cc:1950] T 9fec26011dec4e91868a1cbe71cdf371 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.606118 29087 pending_rounds.cc:77] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316530478754209792 op_type: NO_OP noop_request { }
I20260809 07:58:44.606267 29087 raft_consensus.cc:2889] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 07:58:44.606453 29087 raft_consensus.cc:2272] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.607061 29086 tablet_replica.cc:333] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.607208 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 9fec26011dec4e91868a1cbe71cdf371 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.607293 29086 raft_consensus.cc:2243] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.607573 29086 raft_consensus.cc:2272] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.614297 28788 tablet_service.cc:1511] Processing CreateTablet for tablet fc1ec621e45e4e57b7fb536f3db739eb (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.614764 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1ec621e45e4e57b7fb536f3db739eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.615615 28787 tablet_service.cc:1511] Processing CreateTablet for tablet ab1a97f98f5446daa2523cf10fb8fac4 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
W20260809 07:58:44.616199 28380 consensus_peers.cc:597] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:44.616408 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1a97f98f5446daa2523cf10fb8fac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.618093 28427 tablet_service.cc:1511] Processing CreateTablet for tablet ab1a97f98f5446daa2523cf10fb8fac4 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.618571 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1a97f98f5446daa2523cf10fb8fac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.628350 28620 tablet_service.cc:1511] Processing CreateTablet for tablet ab1a97f98f5446daa2523cf10fb8fac4 (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.628821 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1a97f98f5446daa2523cf10fb8fac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.619637 28931 tablet_bootstrap.cc:492] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.630535 28931 ts_tablet_manager.cc:1403] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.003s
I20260809 07:58:44.631407 28931 raft_consensus.cc:359] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.631580 28931 raft_consensus.cc:385] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.631675 28931 raft_consensus.cc:740] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.592514 29085 ts_tablet_manager.cc:1916] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.632407 28931 consensus_queue.cc:260] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.632644 29086 ts_tablet_manager.cc:1916] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.637506 28931 ts_tablet_manager.cc:1434] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260809 07:58:44.640594 29087 ts_tablet_manager.cc:1916] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.641001 28931 tablet_bootstrap.cc:492] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.643404 28931 tablet_bootstrap.cc:654] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.651777 28614 tablet_service.cc:1511] Processing CreateTablet for tablet fc1ec621e45e4e57b7fb536f3db739eb (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.652318 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1ec621e45e4e57b7fb536f3db739eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.651926 29108 tablet_bootstrap.cc:492] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:44.657856 28931 tablet_bootstrap.cc:492] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.658043 28931 ts_tablet_manager.cc:1403] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.002s
I20260809 07:58:44.658842 28931 raft_consensus.cc:359] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.659070 28931 raft_consensus.cc:385] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.659164 28931 raft_consensus.cc:740] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.659495 28931 consensus_queue.cc:260] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.660022 28931 ts_tablet_manager.cc:1434] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:44.660240 29108 tablet_bootstrap.cc:654] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.660568 28931 tablet_bootstrap.cc:492] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.662420 28931 tablet_bootstrap.cc:654] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.664599 28602 tablet_service.cc:1511] Processing CreateTablet for tablet 550838d288be4ecf8a9dd10aeda6467a (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.665055 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550838d288be4ecf8a9dd10aeda6467a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.671195 29086 ts_tablet_manager.cc:1929] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.671407 29086 log.cc:1199] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/5745cdf610dd4552bfd339a7ed04e95c
I20260809 07:58:44.672032 29086 ts_tablet_manager.cc:1950] T 5745cdf610dd4552bfd339a7ed04e95c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.673534 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 5745cdf610dd4552bfd339a7ed04e95c (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.677479 29087 ts_tablet_manager.cc:1929] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.677708 29087 log.cc:1199] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/437e82d85fc445ce8ae904c2c71d08c7
I20260809 07:58:44.678308 29087 ts_tablet_manager.cc:1950] T 437e82d85fc445ce8ae904c2c71d08c7 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.679770 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 437e82d85fc445ce8ae904c2c71d08c7 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.680145 29087 tablet_replica.cc:333] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.680498 29087 raft_consensus.cc:2243] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.680747 29087 raft_consensus.cc:2272] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.681245 28931 tablet_bootstrap.cc:492] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.681456 28931 ts_tablet_manager.cc:1403] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.017s
I20260809 07:58:44.682487 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 550838d288be4ecf8a9dd10aeda6467a (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.682916 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550838d288be4ecf8a9dd10aeda6467a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.683748 29087 ts_tablet_manager.cc:1916] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.684950 28602 tablet_service.cc:1511] Processing CreateTablet for tablet a48541470e51457d980a8c3ea27b060f (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.685376 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48541470e51457d980a8c3ea27b060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.685585 29086 tablet_replica.cc:333] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.685818 29086 raft_consensus.cc:2243] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.685745 28788 tablet_service.cc:1511] Processing CreateTablet for tablet a48541470e51457d980a8c3ea27b060f (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.686144 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48541470e51457d980a8c3ea27b060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.686205 29086 raft_consensus.cc:2272] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.687770 29086 ts_tablet_manager.cc:1916] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.689814 29001 raft_consensus.cc:493] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.689860 28422 tablet_service.cc:1511] Processing CreateTablet for tablet a48541470e51457d980a8c3ea27b060f (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.689992 29001 raft_consensus.cc:515] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.690367 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48541470e51457d980a8c3ea27b060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.690677 29001 leader_election.cc:290] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.691419 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d0b5e3e19e44f0a2414e7ce3da6d7e" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.692446 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d0b5e3e19e44f0a2414e7ce3da6d7e" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.692679 28462 raft_consensus.cc:2468] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
W20260809 07:58:44.693311 28740 leader_election.cc:343] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:44.696966 28380 consensus_peers.cc:597] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:44.697014 28740 leader_election.cc:304] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:44.697391 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 550838d288be4ecf8a9dd10aeda6467a (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.697504 28941 raft_consensus.cc:493] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.697652 28941 raft_consensus.cc:515] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.697793 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550838d288be4ecf8a9dd10aeda6467a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.698408 28941 leader_election.cc:290] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.698541 29087 ts_tablet_manager.cc:1929] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.698608 28941 raft_consensus.cc:2804] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:44.698697 29087 log.cc:1199] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/5745cdf610dd4552bfd339a7ed04e95c
I20260809 07:58:44.698712 28941 raft_consensus.cc:493] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:44.698814 28941 raft_consensus.cc:3060] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.699112 29087 ts_tablet_manager.cc:1950] T 5745cdf610dd4552bfd339a7ed04e95c P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.700083 29087 tablet_replica.cc:333] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.700212 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 5745cdf610dd4552bfd339a7ed04e95c (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.700300 29087 raft_consensus.cc:2243] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.700616 29087 raft_consensus.cc:2272] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.702625 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f47e81a4dd5943f7873144e373bab017" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.703342 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f47e81a4dd5943f7873144e373bab017" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
W20260809 07:58:44.704110 28740 leader_election.cc:343] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.703779 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.401s user 0.045s sys 0.054s Metrics: {"bytes_written":683,"cfile_cache_hit":16,"cfile_cache_hit_bytes":9236,"cfile_cache_miss":42,"cfile_cache_miss_bytes":14962,"cfile_init":12,"delete_count":0,"delta_iterators_relevant":13,"dirs.queue_time_us":8491,"dirs.run_cpu_time_us":1431,"dirs.run_wall_time_us":11214,"drs_written":1,"lbm_read_time_us":1902,"lbm_reads_lt_1ms":90,"lbm_write_time_us":9897,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":30,"mutex_wait_us":1231,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":37,"spinlock_wait_cycles":249600,"thread_start_us":263032,"threads_started":2,"update_count":16}
W20260809 07:58:44.704309 28740 leader_election.cc:343] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.704438 28740 leader_election.cc:304] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:44.704962 29001 raft_consensus.cc:2749] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:44.706003 29086 ts_tablet_manager.cc:1929] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.706144 29086 log.cc:1199] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/841c52b2284042a8a4dab34f8ed13fe2
I20260809 07:58:44.706512 29095 tablet_bootstrap.cc:492] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:44.706427 28619 tablet_service.cc:1511] Processing CreateTablet for tablet eff7ca7c344b48b685c829148b184b9b (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.706662 29086 ts_tablet_manager.cc:1950] T 841c52b2284042a8a4dab34f8ed13fe2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.706813 29095 ts_tablet_manager.cc:1403] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.114s user 0.007s sys 0.030s
I20260809 07:58:44.706840 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff7ca7c344b48b685c829148b184b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.707788 29086 tablet_replica.cc:333] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.707764 29095 raft_consensus.cc:359] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.707965 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 841c52b2284042a8a4dab34f8ed13fe2 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.708027 29086 raft_consensus.cc:2243] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.708086 29095 raft_consensus.cc:385] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.708210 29086 raft_consensus.cc:2272] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.708252 29095 raft_consensus.cc:740] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.708567 29095 consensus_queue.cc:260] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.709213 28941 raft_consensus.cc:515] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.712013 29095 ts_tablet_manager.cc:1434] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260809 07:58:44.712342 28941 leader_election.cc:290] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.713047 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d0b5e3e19e44f0a2414e7ce3da6d7e" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
W20260809 07:58:44.713660 28740 leader_election.cc:343] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.714221 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d0b5e3e19e44f0a2414e7ce3da6d7e" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:44.714423 28462 raft_consensus.cc:3060] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.715986 28428 tablet_service.cc:1511] Processing CreateTablet for tablet eff7ca7c344b48b685c829148b184b9b (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.716372 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff7ca7c344b48b685c829148b184b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.717290 28462 raft_consensus.cc:2468] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:44.717941 29087 ts_tablet_manager.cc:1916] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.718087 28740 leader_election.cc:304] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:44.718618 28941 raft_consensus.cc:2804] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:44.719332 29086 ts_tablet_manager.cc:1916] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.719389 28941 raft_consensus.cc:697] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:44.720335 29095 tablet_bootstrap.cc:492] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.720360 28941 consensus_queue.cc:237] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.722136 29095 tablet_bootstrap.cc:654] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.722379 28931 raft_consensus.cc:359] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.722575 28931 raft_consensus.cc:385] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.722688 28931 raft_consensus.cc:740] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.722925 28931 consensus_queue.cc:260] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.723503 28788 tablet_service.cc:1511] Processing CreateTablet for tablet eff7ca7c344b48b685c829148b184b9b (DEFAULT_TABLE table=table-1 [id=02e8fe507dcb47419972879b80716181]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:44.723991 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff7ca7c344b48b685c829148b184b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:44.725397 29087 ts_tablet_manager.cc:1929] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.725576 29087 log.cc:1199] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/a2d60a37346b469781732b692cf55c63
I20260809 07:58:44.726094 29087 ts_tablet_manager.cc:1950] T a2d60a37346b469781732b692cf55c63 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.727190 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet a2d60a37346b469781732b692cf55c63 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.727771 28931 ts_tablet_manager.cc:1434] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.046s user 0.001s sys 0.001s
I20260809 07:58:44.728094 29085 ts_tablet_manager.cc:1929] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.729868 28931 tablet_bootstrap.cc:492] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.731523 28931 tablet_bootstrap.cc:654] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.733461 29108 tablet_bootstrap.cc:492] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:44.733703 29108 ts_tablet_manager.cc:1403] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.082s user 0.005s sys 0.000s
I20260809 07:58:44.734555 29108 raft_consensus.cc:359] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.734786 29108 raft_consensus.cc:385] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.734911 29108 raft_consensus.cc:740] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.735244 29108 consensus_queue.cc:260] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.736724 29108 ts_tablet_manager.cc:1434] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:44.737035 29108 tablet_bootstrap.cc:492] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:44.738921 29108 tablet_bootstrap.cc:654] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.747637 29086 ts_tablet_manager.cc:1929] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.747799 29086 log.cc:1199] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/a2d60a37346b469781732b692cf55c63
W20260809 07:58:44.732120 28382 consensus_peers.cc:597] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5745cdf610dd4552bfd339a7ed04e95c. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:44.747884 28149 catalog_manager.cc:5719] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:44.748318 29086 ts_tablet_manager.cc:1950] T a2d60a37346b469781732b692cf55c63 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.732762 29095 tablet_bootstrap.cc:492] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
W20260809 07:58:44.738554 28380 consensus_peers.cc:597] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ac8194b8d8946d797adb804d6420646. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:44.749161 29095 ts_tablet_manager.cc:1403] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.005s
I20260809 07:58:44.750000 29095 raft_consensus.cc:359] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.750222 29095 raft_consensus.cc:385] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.750321 29095 raft_consensus.cc:740] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.750564 29095 consensus_queue.cc:260] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.750783 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet a2d60a37346b469781732b692cf55c63 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.751120 29095 ts_tablet_manager.cc:1434] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:44.751317 29085 log.cc:1199] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/9b93c5f310964747a018b899b0cb2dbf
I20260809 07:58:44.751929 29085 ts_tablet_manager.cc:1950] T 9b93c5f310964747a018b899b0cb2dbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:44.752293 29086 tablet_replica.cc:333] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.752413 29087 tablet_replica.cc:333] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.752499 29086 raft_consensus.cc:2243] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.752632 29087 raft_consensus.cc:2243] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.752820 29087 raft_consensus.cc:2272] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.752693 29086 raft_consensus.cc:2272] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.754330 29087 ts_tablet_manager.cc:1916] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.754411 29086 ts_tablet_manager.cc:1916] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.757773 29085 tablet_replica.cc:333] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:44.757855 29095 tablet_bootstrap.cc:492] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.757962 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 9b93c5f310964747a018b899b0cb2dbf (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.758082 29085 raft_consensus.cc:2243] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.758427 29085 raft_consensus.cc:2272] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.759608 29085 ts_tablet_manager.cc:1916] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.763327 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.763798 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.764119 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.765826 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.766371 29095 tablet_bootstrap.cc:654] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.769121 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.769519 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.769865 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.770196 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.770627 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.770953 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.771272 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.771569 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":5180}
I20260809 07:58:44.771749 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.155217
I20260809 07:58:44.772950 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":225,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.773311 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.155217
I20260809 07:58:44.773696 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.155217
I20260809 07:58:44.774175 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.774434 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.774878 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.775305 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.775424 28961 raft_consensus.cc:493] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.775671 28961 raft_consensus.cc:515] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.776648 28961 leader_election.cc:290] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:44.777688 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d0b5e3e19e44f0a2414e7ce3da6d7e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.777997 28462 raft_consensus.cc:2393] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24137fd377124538a31e92f3b6df0673 in current term 1: Already voted for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in this term.
I20260809 07:58:44.778661 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d0b5e3e19e44f0a2414e7ce3da6d7e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:44.779395 28566 leader_election.cc:304] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:44.779932 28961 raft_consensus.cc:3060] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.780426 29087 ts_tablet_manager.cc:1929] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.780622 29087 log.cc:1199] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/6b81ec52183e4bd28ed4d17ed15f5414
I20260809 07:58:44.781275 29087 ts_tablet_manager.cc:1950] T 6b81ec52183e4bd28ed4d17ed15f5414 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.782493 29085 ts_tablet_manager.cc:1929] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.784237 29086 ts_tablet_manager.cc:1929] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.784456 29085 log.cc:1199] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/8ac8194b8d8946d797adb804d6420646
I20260809 07:58:44.786036 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.005s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":197,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.791703 29095 tablet_bootstrap.cc:492] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:44.791988 29095 ts_tablet_manager.cc:1403] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.005s
I20260809 07:58:44.793059 29095 raft_consensus.cc:359] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.793423 29095 raft_consensus.cc:385] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.793588 29095 raft_consensus.cc:740] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.794059 29095 consensus_queue.cc:260] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.795902 29095 ts_tablet_manager.cc:1434] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:44.796232 29095 tablet_bootstrap.cc:492] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.784158 28961 raft_consensus.cc:2749] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:44.798002 29087 tablet_replica.cc:333] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.798013 29095 tablet_bootstrap.cc:654] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.798188 29087 raft_consensus.cc:2243] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.798224 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 6b81ec52183e4bd28ed4d17ed15f5414 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.798398 29087 raft_consensus.cc:2272] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.801127 29085 ts_tablet_manager.cc:1950] T 8ac8194b8d8946d797adb804d6420646 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:44.785461 28931 tablet_bootstrap.cc:492] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
W20260809 07:58:44.794417 28740 consensus_peers.cc:597] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:44.801621 28931 ts_tablet_manager.cc:1403] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.072s user 0.002s sys 0.009s
I20260809 07:58:44.802475 28931 raft_consensus.cc:359] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.802758 28931 raft_consensus.cc:385] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.802875 28931 raft_consensus.cc:740] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.802875 29087 ts_tablet_manager.cc:1916] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.796859 29108 tablet_bootstrap.cc:492] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:44.803205 29108 ts_tablet_manager.cc:1403] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.066s user 0.012s sys 0.031s
I20260809 07:58:44.803344 28931 consensus_queue.cc:260] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.804239 29108 raft_consensus.cc:359] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.807623 29108 raft_consensus.cc:385] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.807787 29108 raft_consensus.cc:740] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.807842 28931 ts_tablet_manager.cc:1434] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 07:58:44.808163 29108 consensus_queue.cc:260] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.810864 29108 ts_tablet_manager.cc:1434] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260809 07:58:44.812373 29108 tablet_bootstrap.cc:492] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:44.813148 28931 tablet_bootstrap.cc:492] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.814940 28931 tablet_bootstrap.cc:654] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.814951 29108 tablet_bootstrap.cc:654] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.812817 29095 tablet_bootstrap.cc:492] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:44.816025 29095 ts_tablet_manager.cc:1403] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.004s
I20260809 07:58:44.816825 29095 raft_consensus.cc:359] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.817055 29095 raft_consensus.cc:385] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.817165 29095 raft_consensus.cc:740] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.817444 29095 consensus_queue.cc:260] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.817968 29095 ts_tablet_manager.cc:1434] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:44.818254 29095 tablet_bootstrap.cc:492] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.820019 29095 tablet_bootstrap.cc:654] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.820966 29086 log.cc:1199] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/b95825d687ab41878ef822421d853385
I20260809 07:58:44.821570 29086 ts_tablet_manager.cc:1950] T b95825d687ab41878ef822421d853385 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.822569 29086 tablet_replica.cc:333] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.822702 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet b95825d687ab41878ef822421d853385 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.822778 29086 raft_consensus.cc:2243] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.823096 29086 raft_consensus.cc:2272] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.823897 29086 ts_tablet_manager.cc:1916] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.824194 29085 tablet_replica.cc:333] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:44.824245 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 8ac8194b8d8946d797adb804d6420646 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.824397 29085 raft_consensus.cc:2243] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.824571 29087 ts_tablet_manager.cc:1929] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.824724 29087 log.cc:1199] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/b1365599d9764ccfbfb0cb954f83de69
I20260809 07:58:44.825198 29087 ts_tablet_manager.cc:1950] T b1365599d9764ccfbfb0cb954f83de69 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.825565 29085 raft_consensus.cc:2272] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.826246 29087 tablet_replica.cc:333] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.826429 29087 raft_consensus.cc:2243] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.826740 29087 raft_consensus.cc:2272] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.827122 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet b1365599d9764ccfbfb0cb954f83de69 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.828097 28931 tablet_bootstrap.cc:492] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.828235 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.054s user 0.009s sys 0.004s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":8675,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":88,"lbm_read_time_us":442,"lbm_reads_lt_1ms":23,"lbm_write_time_us":396,"lbm_writes_lt_1ms":9,"mutex_wait_us":11980,"peak_mem_usage":32768,"reinsert_count":0,"update_count":17}
I20260809 07:58:44.828739 29085 ts_tablet_manager.cc:1916] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.828877 29108 tablet_bootstrap.cc:492] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:44.829102 29108 ts_tablet_manager.cc:1403] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.001s
I20260809 07:58:44.829496 28931 ts_tablet_manager.cc:1403] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260809 07:58:44.830121 29108 raft_consensus.cc:359] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.830365 29108 raft_consensus.cc:385] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.830372 28931 raft_consensus.cc:359] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.830587 28931 raft_consensus.cc:385] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.830623 29095 tablet_bootstrap.cc:492] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:44.830521 29108 raft_consensus.cc:740] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.830689 28931 raft_consensus.cc:740] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.830770 29095 ts_tablet_manager.cc:1403] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260809 07:58:44.831045 29108 consensus_queue.cc:260] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.831607 29086 ts_tablet_manager.cc:1929] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.831549 29095 raft_consensus.cc:359] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.831724 29095 raft_consensus.cc:385] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.831751 29086 log.cc:1199] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/9bd831b22d6044209063ec4b9a984e6d
I20260809 07:58:44.831800 29095 raft_consensus.cc:740] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.832085 29095 consensus_queue.cc:260] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.832196 29086 ts_tablet_manager.cc:1950] T 9bd831b22d6044209063ec4b9a984e6d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.833297 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 9bd831b22d6044209063ec4b9a984e6d (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.834074 29087 ts_tablet_manager.cc:1916] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.835059 29108 ts_tablet_manager.cc:1434] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 07:58:44.835348 29095 ts_tablet_manager.cc:1434] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 07:58:44.835806 28931 consensus_queue.cc:260] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.838111 29095 tablet_bootstrap.cc:492] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.839389 28931 ts_tablet_manager.cc:1434] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260809 07:58:44.840034 29095 tablet_bootstrap.cc:654] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.840951 29087 ts_tablet_manager.cc:1929] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.841111 29087 log.cc:1199] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/392548585cf1446f99dbc268831cae8e
I20260809 07:58:44.841616 29108 tablet_bootstrap.cc:492] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:44.841755 29087 ts_tablet_manager.cc:1950] T 392548585cf1446f99dbc268831cae8e P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.842111 29001 raft_consensus.cc:493] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.842281 29001 raft_consensus.cc:515] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.842826 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 392548585cf1446f99dbc268831cae8e (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.842940 29086 tablet_replica.cc:333] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.843149 29086 raft_consensus.cc:2243] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.843381 29086 raft_consensus.cc:2272] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.843557 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39664b79fe3246dfbae1cb687bfc8129" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.843757 28462 raft_consensus.cc:2468] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:44.843817 29108 tablet_bootstrap.cc:654] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.843819 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39664b79fe3246dfbae1cb687bfc8129" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.844300 28740 leader_election.cc:304] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:44.844417 29086 ts_tablet_manager.cc:1916] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.842964 29001 leader_election.cc:290] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.844784 29003 raft_consensus.cc:2804] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260809 07:58:44.844681 28740 leader_election.cc:343] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.844939 29003 raft_consensus.cc:493] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:44.845093 29003 raft_consensus.cc:3060] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.848029 28931 tablet_bootstrap.cc:492] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.849531 29003 raft_consensus.cc:515] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.850004 29001 raft_consensus.cc:493] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.850183 29001 raft_consensus.cc:515] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.850860 29001 leader_election.cc:290] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.850857 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39664b79fe3246dfbae1cb687bfc8129" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:44.851046 28462 raft_consensus.cc:3060] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.851979 29086 ts_tablet_manager.cc:1929] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.852103 29086 log.cc:1199] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/392548585cf1446f99dbc268831cae8e
I20260809 07:58:44.852540 28931 tablet_bootstrap.cc:654] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.852605 29108 tablet_bootstrap.cc:492] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:44.852694 29086 ts_tablet_manager.cc:1950] T 392548585cf1446f99dbc268831cae8e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.852783 29108 ts_tablet_manager.cc:1403] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260809 07:58:44.853572 29087 tablet_replica.cc:333] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.853545 29108 raft_consensus.cc:359] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.853782 29087 raft_consensus.cc:2243] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.853758 29108 raft_consensus.cc:385] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.853940 29108 raft_consensus.cc:740] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.854033 29087 raft_consensus.cc:2272] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.854058 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6265b1b590cc40739365212572bfc6ed" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.854162 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6265b1b590cc40739365212572bfc6ed" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.854286 28648 raft_consensus.cc:2468] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:44.854624 29086 tablet_replica.cc:333] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
W20260809 07:58:44.854705 28740 leader_election.cc:343] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.854811 28462 raft_consensus.cc:2468] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:44.854861 29086 raft_consensus.cc:2243] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.854995 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 392548585cf1446f99dbc268831cae8e (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.855039 29086 raft_consensus.cc:2272] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.855078 29108 consensus_queue.cc:260] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.855232 28740 leader_election.cc:304] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:44.855545 29001 raft_consensus.cc:2804] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:44.855703 29001 raft_consensus.cc:697] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:44.855798 29087 ts_tablet_manager.cc:1916] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.855932 29001 consensus_queue.cc:237] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.857498 29003 leader_election.cc:290] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.857831 29108 ts_tablet_manager.cc:1434] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:58:44.858407 28740 leader_election.cc:304] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:44.860579 29001 raft_consensus.cc:2804] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:44.860673 29001 raft_consensus.cc:493] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:44.860402 28149 catalog_manager.cc:5719] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:44.860769 29001 raft_consensus.cc:3060] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.863715 29001 raft_consensus.cc:515] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.864431 29001 leader_election.cc:290] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.865533 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6265b1b590cc40739365212572bfc6ed" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
W20260809 07:58:44.866204 28740 leader_election.cc:343] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.867537 29001 raft_consensus.cc:493] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.867741 29001 raft_consensus.cc:515] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.868749 29001 leader_election.cc:290] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.869323 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75642018d47b45e6ba74466c6d5ee95e" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:44.869596 28462 raft_consensus.cc:2468] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:44.870214 28740 leader_election.cc:304] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:44.870329 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39664b79fe3246dfbae1cb687bfc8129" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:44.870617 29001 raft_consensus.cc:2804] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:44.870679 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.870759 29001 raft_consensus.cc:493] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:44.870872 29001 raft_consensus.cc:3060] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.871012 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.871315 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.871608 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.871630 29086 ts_tablet_manager.cc:1916] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.872010 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.872294 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.872529 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.873663 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6265b1b590cc40739365212572bfc6ed" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
W20260809 07:58:44.874073 28740 leader_election.cc:343] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.874436 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75642018d47b45e6ba74466c6d5ee95e" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.874536 29001 raft_consensus.cc:515] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.874816 28648 raft_consensus.cc:3060] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.875531 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75642018d47b45e6ba74466c6d5ee95e" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:44.875711 28462 raft_consensus.cc:3060] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.877285 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.877611 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
W20260809 07:58:44.877681 28740 leader_election.cc:343] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.877772 28648 raft_consensus.cc:2468] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:44.877869 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.878079 29108 tablet_bootstrap.cc:492] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:44.878144 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.878185 28740 leader_election.cc:304] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:44.878785 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.002s sys 0.003s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":171,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.880035 29108 tablet_bootstrap.cc:654] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.880110 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.880442 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.880705 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.881275 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.881594 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.881945 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.882050 29087 ts_tablet_manager.cc:1929] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.882184 29087 log.cc:1199] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/9fec26011dec4e91868a1cbe71cdf371
I20260809 07:58:44.882452 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.882709 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.882779 29087 ts_tablet_manager.cc:1950] T 9fec26011dec4e91868a1cbe71cdf371 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.883270 29003 raft_consensus.cc:2804] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:44.883805 29087 tablet_replica.cc:333] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.884018 29087 raft_consensus.cc:2243] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.884008 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 9fec26011dec4e91868a1cbe71cdf371 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.884207 29087 raft_consensus.cc:2272] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.884205 29003 raft_consensus.cc:697] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:44.884728 29003 consensus_queue.cc:237] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.887040 29001 leader_election.cc:290] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:44.890367 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75642018d47b45e6ba74466c6d5ee95e" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:44.890934 29014 raft_consensus.cc:493] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.891170 29014 raft_consensus.cc:515] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.892076 29014 leader_election.cc:290] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:44.893059 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c44aace20a640ef8aca975a73af0d93" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
W20260809 07:58:44.893989 28380 leader_election.cc:343] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:44.894732 28740 leader_election.cc:343] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.894749 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c44aace20a640ef8aca975a73af0d93" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:44.894937 29086 ts_tablet_manager.cc:1929] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.895049 28826 raft_consensus.cc:2468] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:44.896137 29087 ts_tablet_manager.cc:1916] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.896986 28382 leader_election.cc:304] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:44.897583 29014 raft_consensus.cc:2804] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:44.897691 29014 raft_consensus.cc:493] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:44.897806 29014 raft_consensus.cc:3060] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.898844 28149 catalog_manager.cc:5719] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:44.895090 29086 log.cc:1199] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/acb842662fa640ee932f06fd7b5b7fc2
I20260809 07:58:44.902209 29086 ts_tablet_manager.cc:1950] T acb842662fa640ee932f06fd7b5b7fc2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.903357 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet acb842662fa640ee932f06fd7b5b7fc2 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.904492 29086 tablet_replica.cc:333] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.904713 29086 raft_consensus.cc:2243] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:44.904968 28462 raft_consensus.cc:2468] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:44.905125 29086 raft_consensus.cc:2272] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.906764 29085 ts_tablet_manager.cc:1929] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.907296 29095 tablet_bootstrap.cc:492] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:44.907323 29085 log.cc:1199] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/acb842662fa640ee932f06fd7b5b7fc2
I20260809 07:58:44.907466 29095 ts_tablet_manager.cc:1403] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.069s user 0.000s sys 0.009s
I20260809 07:58:44.908077 29085 ts_tablet_manager.cc:1950] T acb842662fa640ee932f06fd7b5b7fc2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:44.908344 29095 raft_consensus.cc:359] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.908522 29095 raft_consensus.cc:385] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.908633 29095 raft_consensus.cc:740] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.908699 29086 ts_tablet_manager.cc:1916] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.908902 29095 consensus_queue.cc:260] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.909472 29095 ts_tablet_manager.cc:1434] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 07:58:44.910538 29095 tablet_bootstrap.cc:492] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.911056 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.911574 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.912015 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.912266 29095 tablet_bootstrap.cc:654] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.912358 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.912657 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.914252 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.914341 28740 leader_election.cc:304] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:44.914516 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.917151 29003 raft_consensus.cc:2804] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:44.917318 29003 raft_consensus.cc:697] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:44.917512 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.917614 29003 consensus_queue.cc:237] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.918785 29087 ts_tablet_manager.cc:1929] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.918944 29087 log.cc:1199] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/9bd831b22d6044209063ec4b9a984e6d
I20260809 07:58:44.919463 29087 ts_tablet_manager.cc:1950] T 9bd831b22d6044209063ec4b9a984e6d P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.920840 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 9bd831b22d6044209063ec4b9a984e6d (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.921078 29014 raft_consensus.cc:515] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.922236 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c44aace20a640ef8aca975a73af0d93" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
W20260809 07:58:44.922801 28380 leader_election.cc:343] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:44.923626 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.004s sys 0.002s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":168,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.923713 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c44aace20a640ef8aca975a73af0d93" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:44.923957 28826 raft_consensus.cc:3060] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:44.924149 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.924350 29014 leader_election.cc:290] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:44.924522 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":163,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.926810 29087 tablet_replica.cc:333] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.927011 29087 raft_consensus.cc:2243] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.927230 29087 raft_consensus.cc:2272] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.927354 28826 raft_consensus.cc:2468] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:44.927654 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":670,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":64,"lbm_read_time_us":187,"lbm_reads_lt_1ms":8,"lbm_write_time_us":194,"lbm_writes_lt_1ms":4,"mutex_wait_us":1387,"reinsert_count":0,"update_count":2}
I20260809 07:58:44.930776 28149 catalog_manager.cc:5719] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:44.931460 29095 tablet_bootstrap.cc:492] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:44.931670 29095 ts_tablet_manager.cc:1403] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.004s
I20260809 07:58:44.932103 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":18995}
I20260809 07:58:44.932677 29095 raft_consensus.cc:359] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.932902 29095 raft_consensus.cc:385] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.933009 29095 raft_consensus.cc:740] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.934361 29087 ts_tablet_manager.cc:1916] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.935194 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet acb842662fa640ee932f06fd7b5b7fc2 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.935559 29085 tablet_replica.cc:333] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:44.935807 29085 raft_consensus.cc:2243] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.936007 29085 raft_consensus.cc:2272] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.936537 28382 leader_election.cc:304] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:44.937076 29085 ts_tablet_manager.cc:1916] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.938374 28931 tablet_bootstrap.cc:492] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.938571 28931 ts_tablet_manager.cc:1403] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.000s
I20260809 07:58:44.933300 29095 consensus_queue.cc:260] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.938835 28989 raft_consensus.cc:2804] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:44.939020 28989 raft_consensus.cc:697] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:44.939162 29095 ts_tablet_manager.cc:1434] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 07:58:44.939306 28989 consensus_queue.cc:237] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.939503 29095 tablet_bootstrap.cc:492] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:44.941596 29095 tablet_bootstrap.cc:654] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.945835 29086 ts_tablet_manager.cc:1929] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.945648 28149 catalog_manager.cc:5719] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:44.945977 29086 log.cc:1199] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/6b81ec52183e4bd28ed4d17ed15f5414
I20260809 07:58:44.946537 29086 ts_tablet_manager.cc:1950] T 6b81ec52183e4bd28ed4d17ed15f5414 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:44.946813 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.947060 29085 ts_tablet_manager.cc:1929] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.947191 29085 log.cc:1199] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/6eac9128f38545f09d093eed3c9839ae
I20260809 07:58:44.947604 29085 ts_tablet_manager.cc:1950] T 6eac9128f38545f09d093eed3c9839ae P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:44.947767 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 6b81ec52183e4bd28ed4d17ed15f5414 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.948761 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":166,"lbm_writes_lt_1ms":4,"mutex_wait_us":14778,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.948968 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.949287 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.949595 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.949875 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.949662 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.067s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":670,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":1509,"lbm_read_time_us":217,"lbm_reads_lt_1ms":8,"lbm_write_time_us":200,"lbm_writes_lt_1ms":4,"mutex_wait_us":3098,"reinsert_count":0,"update_count":2}
I20260809 07:58:44.950450 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.127970
I20260809 07:58:44.950845 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.127970
I20260809 07:58:44.951321 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.951675 29086 tablet_replica.cc:333] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:44.951802 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.951934 29086 raft_consensus.cc:2243] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.952122 29086 raft_consensus.cc:2272] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.952183 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.952410 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.952777 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.002s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":175,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.954170 29086 ts_tablet_manager.cc:1916] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.955564 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.002s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":168,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.955986 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":31262}
I20260809 07:58:44.957660 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.957922 29087 ts_tablet_manager.cc:1929] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.957984 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.958078 29087 log.cc:1199] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/841c52b2284042a8a4dab34f8ed13fe2
I20260809 07:58:44.958285 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.958582 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.958636 29087 ts_tablet_manager.cc:1950] T 841c52b2284042a8a4dab34f8ed13fe2 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:44.961076 28931 raft_consensus.cc:359] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.961376 28931 raft_consensus.cc:385] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.961478 28931 raft_consensus.cc:740] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.961516 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.961720 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.961767 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.961820 28931 consensus_queue.cc:260] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.962000 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:44.962337 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.962366 28931 ts_tablet_manager.cc:1434] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.024s user 0.001s sys 0.002s
I20260809 07:58:44.962415 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.962666 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.962921 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.962934 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:44.967504 28931 tablet_bootstrap.cc:492] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.969521 28931 tablet_bootstrap.cc:654] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.971724 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.002s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":694,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":3593,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":559,"lbm_read_time_us":144,"lbm_reads_lt_1ms":8,"lbm_write_time_us":147,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260809 07:58:44.972602 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:44.975274 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":5155}
I20260809 07:58:44.976125 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":5517}
I20260809 07:58:44.975242 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"mutex_wait_us":7181,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.968087 29014 raft_consensus.cc:493] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.979758 28989 raft_consensus.cc:493] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:44.979943 28989 raft_consensus.cc:515] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.981561 29108 tablet_bootstrap.cc:492] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:44.981737 29108 ts_tablet_manager.cc:1403] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.123s user 0.021s sys 0.020s
I20260809 07:58:44.982589 29108 raft_consensus.cc:359] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.982776 29108 raft_consensus.cc:385] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.982853 29108 raft_consensus.cc:740] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.983085 29108 consensus_queue.cc:260] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.983610 29108 ts_tablet_manager.cc:1434] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:44.984368 28931 tablet_bootstrap.cc:492] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:44.984509 28931 ts_tablet_manager.cc:1403] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.009s
I20260809 07:58:44.984441 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.004s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":715,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":59,"lbm_read_time_us":219,"lbm_reads_lt_1ms":12,"lbm_write_time_us":136,"lbm_writes_lt_1ms":4,"mutex_wait_us":3650,"reinsert_count":0,"update_count":4}
I20260809 07:58:44.985299 28931 raft_consensus.cc:359] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.985553 28931 raft_consensus.cc:385] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:44.985224 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.011s sys 0.000s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":12435,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":89,"dirs.run_cpu_time_us":246,"dirs.run_wall_time_us":1817,"lbm_read_time_us":585,"lbm_reads_lt_1ms":21,"lbm_write_time_us":326,"lbm_writes_lt_1ms":9,"mutex_wait_us":4952,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":389376,"update_count":4}
I20260809 07:58:44.985678 28931 raft_consensus.cc:740] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:44.985788 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":21374}
I20260809 07:58:44.985620 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":21260}
I20260809 07:58:44.986071 28931 consensus_queue.cc:260] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.986552 29108 tablet_bootstrap.cc:492] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:44.987713 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 841c52b2284042a8a4dab34f8ed13fe2 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:44.987942 28989 leader_election.cc:290] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:44.988080 29087 tablet_replica.cc:333] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:44.988317 29087 raft_consensus.cc:2243] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:44.988523 29087 raft_consensus.cc:2272] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:44.988948 29108 tablet_bootstrap.cc:654] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.989300 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f47e81a4dd5943f7873144e373bab017" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:44.989544 28648 raft_consensus.cc:2468] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:44.989626 29087 ts_tablet_manager.cc:1916] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:44.989714 28931 ts_tablet_manager.cc:1434] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 07:58:44.990309 29014 raft_consensus.cc:515] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:44.990442 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f47e81a4dd5943f7873144e373bab017" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:44.990618 28826 raft_consensus.cc:2468] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:44.991083 28380 leader_election.cc:304] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:44.992718 28931 tablet_bootstrap.cc:492] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:44.995481 28931 tablet_bootstrap.cc:654] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:44.996989 29087 ts_tablet_manager.cc:1929] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:44.997184 29087 log.cc:1199] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/b95825d687ab41878ef822421d853385
I20260809 07:58:44.986047 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.001s sys 0.004s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":146,"lbm_writes_lt_1ms":4,"mutex_wait_us":17069,"reinsert_count":0,"update_count":1}
I20260809 07:58:44.999624 28998 raft_consensus.cc:2804] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:44.999748 28998 raft_consensus.cc:493] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:44.999909 29086 ts_tablet_manager.cc:1929] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.000082 29086 log.cc:1199] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/b1365599d9764ccfbfb0cb954f83de69
I20260809 07:58:45.000422 28998 raft_consensus.cc:3060] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.000577 29086 ts_tablet_manager.cc:1950] T b1365599d9764ccfbfb0cb954f83de69 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:45.001889 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet b1365599d9764ccfbfb0cb954f83de69 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.002585 29014 leader_election.cc:290] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
W20260809 07:58:45.003091 28382 consensus_peers.cc:597] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1365599d9764ccfbfb0cb954f83de69. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:45.003137 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 6eac9128f38545f09d093eed3c9839ae (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.003764 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6265b1b590cc40739365212572bfc6ed" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.004683 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6265b1b590cc40739365212572bfc6ed" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.004910 28648 raft_consensus.cc:2393] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in current term 1: Already voted for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in this term.
I20260809 07:58:45.005328 28380 leader_election.cc:304] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.006790 29095 tablet_bootstrap.cc:492] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:45.006971 29095 ts_tablet_manager.cc:1403] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.003s
I20260809 07:58:45.007539 29087 ts_tablet_manager.cc:1950] T b95825d687ab41878ef822421d853385 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:45.008090 29014 raft_consensus.cc:3060] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.008229 29108 tablet_bootstrap.cc:492] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.008380 29108 ts_tablet_manager.cc:1403] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.000s
I20260809 07:58:45.009097 29108 raft_consensus.cc:359] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.009315 29108 raft_consensus.cc:385] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.009430 29108 raft_consensus.cc:740] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.009724 29108 consensus_queue.cc:260] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.010049 28931 tablet_bootstrap.cc:492] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:45.010229 28931 ts_tablet_manager.cc:1403] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.003s
I20260809 07:58:45.010417 29108 ts_tablet_manager.cc:1434] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:45.010510 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet b95825d687ab41878ef822421d853385 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.010717 29108 tablet_bootstrap.cc:492] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.011034 28931 raft_consensus.cc:359] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.011226 28931 raft_consensus.cc:385] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.011308 28931 raft_consensus.cc:740] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.011641 28931 consensus_queue.cc:260] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.012184 28931 ts_tablet_manager.cc:1434] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:45.012574 28931 tablet_bootstrap.cc:492] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:45.014803 28931 tablet_bootstrap.cc:654] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.016158 29095 raft_consensus.cc:359] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.016419 29095 raft_consensus.cc:385] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.016526 29095 raft_consensus.cc:740] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.016840 29095 consensus_queue.cc:260] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.017490 29108 tablet_bootstrap.cc:654] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.017546 29095 ts_tablet_manager.cc:1434] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260809 07:58:45.017915 29095 tablet_bootstrap.cc:492] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:45.024796 28998 raft_consensus.cc:515] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.027668 29108 tablet_bootstrap.cc:492] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.027822 29108 ts_tablet_manager.cc:1403] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.008s
I20260809 07:58:45.027962 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f47e81a4dd5943f7873144e373bab017" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:45.028244 28648 raft_consensus.cc:3060] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.028596 29108 raft_consensus.cc:359] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.028770 29108 raft_consensus.cc:385] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.028868 29108 raft_consensus.cc:740] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.029179 29108 consensus_queue.cc:260] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.030722 29108 ts_tablet_manager.cc:1434] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:45.031010 29108 tablet_bootstrap.cc:492] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.032503 28648 raft_consensus.cc:2468] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:45.032846 29108 tablet_bootstrap.cc:654] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.037392 28380 leader_election.cc:304] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:45.037802 28942 raft_consensus.cc:2804] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.037992 28942 raft_consensus.cc:697] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:45.038333 28942 consensus_queue.cc:237] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.039820 28961 raft_consensus.cc:493] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.040001 28961 raft_consensus.cc:515] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.040647 28961 leader_election.cc:290] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.041564 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33198e831822470db98234b54d09c6bc" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.041822 28462 raft_consensus.cc:2468] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:45.042498 29108 tablet_bootstrap.cc:492] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.042652 29108 ts_tablet_manager.cc:1403] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260809 07:58:45.042688 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33198e831822470db98234b54d09c6bc" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.042758 28565 leader_election.cc:304] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:45.042922 28826 raft_consensus.cc:2468] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:45.043110 28961 raft_consensus.cc:2804] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.043227 28961 raft_consensus.cc:493] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.043371 28961 raft_consensus.cc:3060] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.043460 29108 raft_consensus.cc:359] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.043645 29108 raft_consensus.cc:385] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.043712 29108 raft_consensus.cc:740] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.043973 29108 consensus_queue.cc:260] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.046444 29108 ts_tablet_manager.cc:1434] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 07:58:45.049321 29108 tablet_bootstrap.cc:492] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.050201 28961 raft_consensus.cc:515] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.051259 29108 tablet_bootstrap.cc:654] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.052119 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33198e831822470db98234b54d09c6bc" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:45.052384 28462 raft_consensus.cc:3060] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.053815 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33198e831822470db98234b54d09c6bc" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:45.054028 28826 raft_consensus.cc:3060] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.056210 28462 raft_consensus.cc:2468] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:45.056627 29108 tablet_bootstrap.cc:492] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.056783 29108 ts_tablet_manager.cc:1403] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 07:58:45.056783 28565 leader_election.cc:304] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:45.057199 29029 raft_consensus.cc:2804] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.057262 28826 raft_consensus.cc:2468] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:45.057396 29029 raft_consensus.cc:697] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:45.057886 29029 consensus_queue.cc:237] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.058045 29108 raft_consensus.cc:359] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.058215 29108 raft_consensus.cc:385] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.058301 29108 raft_consensus.cc:740] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.058575 29108 consensus_queue.cc:260] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.059216 29108 ts_tablet_manager.cc:1434] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:45.059495 29108 tablet_bootstrap.cc:492] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.061367 29108 tablet_bootstrap.cc:654] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.067394 29014 raft_consensus.cc:2749] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:45.068598 28998 leader_election.cc:290] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.069155 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f47e81a4dd5943f7873144e373bab017" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:45.069358 28826 raft_consensus.cc:3060] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.070189 28961 leader_election.cc:290] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.070779 29108 tablet_bootstrap.cc:492] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.070935 29108 ts_tablet_manager.cc:1403] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.007s
I20260809 07:58:45.071593 29108 raft_consensus.cc:359] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.071794 29108 raft_consensus.cc:385] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.071880 29108 raft_consensus.cc:740] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.072099 29108 consensus_queue.cc:260] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.072536 29108 ts_tablet_manager.cc:1434] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:45.072589 28826 raft_consensus.cc:2468] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:45.072832 29108 tablet_bootstrap.cc:492] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.074656 29108 tablet_bootstrap.cc:654] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.075199 29095 tablet_bootstrap.cc:654] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.080122 29108 tablet_bootstrap.cc:492] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.080271 29108 ts_tablet_manager.cc:1403] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260809 07:58:45.081315 29108 raft_consensus.cc:359] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.081547 29108 raft_consensus.cc:385] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.081643 29108 raft_consensus.cc:740] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.081929 29108 consensus_queue.cc:260] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.082517 29108 ts_tablet_manager.cc:1434] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:45.082791 29108 tablet_bootstrap.cc:492] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.084439 29108 tablet_bootstrap.cc:654] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.089238 29108 tablet_bootstrap.cc:492] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.089442 29108 ts_tablet_manager.cc:1403] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260809 07:58:45.090122 29108 raft_consensus.cc:359] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.090317 29108 raft_consensus.cc:385] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.090382 29108 raft_consensus.cc:740] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.090633 29108 consensus_queue.cc:260] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.091163 29108 ts_tablet_manager.cc:1434] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:45.091445 29108 tablet_bootstrap.cc:492] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.093271 29108 tablet_bootstrap.cc:654] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.097496 28989 raft_consensus.cc:493] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.097708 28989 raft_consensus.cc:515] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.098532 28989 leader_election.cc:290] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.100085 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193f851a4c1043e1af5aaf4b7190730d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.100330 28826 raft_consensus.cc:2468] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:45.103214 29108 tablet_bootstrap.cc:492] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.103340 29108 ts_tablet_manager.cc:1403] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260809 07:58:45.104249 29108 raft_consensus.cc:359] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.104461 29108 raft_consensus.cc:385] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.104559 29108 raft_consensus.cc:740] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.104934 29108 consensus_queue.cc:260] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.105695 29108 ts_tablet_manager.cc:1434] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:45.106004 29108 tablet_bootstrap.cc:492] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.106142 28382 leader_election.cc:304] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.107048 28989 raft_consensus.cc:2804] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.107172 28989 raft_consensus.cc:493] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.107296 28989 raft_consensus.cc:3060] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.107638 29108 tablet_bootstrap.cc:654] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.109754 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193f851a4c1043e1af5aaf4b7190730d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.110075 28648 raft_consensus.cc:2468] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:45.113000 28989 raft_consensus.cc:515] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.115239 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193f851a4c1043e1af5aaf4b7190730d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:45.115226 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193f851a4c1043e1af5aaf4b7190730d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:45.115518 28826 raft_consensus.cc:3060] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.115536 28648 raft_consensus.cc:3060] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.118988 28826 raft_consensus.cc:2468] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:45.119233 28648 raft_consensus.cc:2468] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:45.119817 28380 leader_election.cc:304] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:45.120651 29014 raft_consensus.cc:2804] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.120868 29014 raft_consensus.cc:697] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:45.121318 29014 consensus_queue.cc:237] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.123633 29003 raft_consensus.cc:493] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.123853 29003 raft_consensus.cc:515] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.124670 29003 leader_election.cc:290] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:45.125763 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a54a36e22474698a198d8c25bafa16b" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.126041 28647 raft_consensus.cc:2468] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:45.127172 28740 leader_election.cc:304] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.127565 29003 raft_consensus.cc:2804] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.127704 29003 raft_consensus.cc:493] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.127816 29003 raft_consensus.cc:3060] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.132323 29003 raft_consensus.cc:515] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.133391 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a54a36e22474698a198d8c25bafa16b" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.133641 28462 raft_consensus.cc:2468] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:45.135033 29003 leader_election.cc:290] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:45.135643 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a54a36e22474698a198d8c25bafa16b" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:45.136097 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a54a36e22474698a198d8c25bafa16b" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
W20260809 07:58:45.136245 28382 consensus_peers.cc:597] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b95825d687ab41878ef822421d853385. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:45.136344 28647 raft_consensus.cc:3060] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.137636 28462 raft_consensus.cc:3060] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.140276 28647 raft_consensus.cc:2468] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:45.141103 28740 leader_election.cc:304] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.141621 29003 raft_consensus.cc:2804] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.142848 29003 raft_consensus.cc:697] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:45.142963 29085 tablet_replica.cc:333] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.143453 29085 raft_consensus.cc:2243] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:45.143695 29085 pending_rounds.cc:70] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Trying to abort 1 pending ops.
I20260809 07:58:45.143910 29085 pending_rounds.cc:77] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316530478128066560 op_type: NO_OP noop_request { }
I20260809 07:58:45.144364 29085 raft_consensus.cc:2889] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 07:58:45.144546 29085 raft_consensus.cc:2272] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.145134 28989 leader_election.cc:290] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.146222 29085 ts_tablet_manager.cc:1916] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.150640 29003 consensus_queue.cc:237] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.150889 28462 raft_consensus.cc:2468] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:45.159003 28149 catalog_manager.cc:5719] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:45.159741 28151 catalog_manager.cc:5719] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:45.175120 28931 tablet_bootstrap.cc:492] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:45.175292 28931 ts_tablet_manager.cc:1403] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.163s user 0.042s sys 0.094s
I20260809 07:58:45.176054 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.176405 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.176843 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.177142 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.177464 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.167404
I20260809 07:58:45.177753 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.167404
I20260809 07:58:45.177984 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.178422 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.179066 28931 raft_consensus.cc:359] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.179271 28931 raft_consensus.cc:385] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.179358 28931 raft_consensus.cc:740] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.179637 28931 consensus_queue.cc:260] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.179956 29108 tablet_bootstrap.cc:492] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.180163 29108 ts_tablet_manager.cc:1403] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.074s user 0.010s sys 0.042s
I20260809 07:58:45.181119 29108 raft_consensus.cc:359] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.181432 29108 raft_consensus.cc:385] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.181535 29108 raft_consensus.cc:740] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.181881 29108 consensus_queue.cc:260] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.184056 28931 ts_tablet_manager.cc:1434] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.009s user 0.001s sys 0.002s
I20260809 07:58:45.185974 28931 tablet_bootstrap.cc:492] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:45.187016 29108 ts_tablet_manager.cc:1434] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260809 07:58:45.188762 28931 tablet_bootstrap.cc:654] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
W20260809 07:58:45.188970 28380 consensus_peers.cc:597] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b95825d687ab41878ef822421d853385. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:45.189518 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.189854 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.189965 29108 tablet_bootstrap.cc:492] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.190162 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.190402 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.192183 29108 tablet_bootstrap.cc:654] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.192221 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.192516 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.192796 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.193249 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.174756
I20260809 07:58:45.193584 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.174756
I20260809 07:58:45.193899 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.174756
I20260809 07:58:45.194278 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.194540 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.194913 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.195329 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.195654 28149 catalog_manager.cc:5719] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:45.196235 28149 catalog_manager.cc:5719] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:45.201817 29108 tablet_bootstrap.cc:492] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.202010 29108 ts_tablet_manager.cc:1403] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260809 07:58:45.202982 29108 raft_consensus.cc:359] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.203307 29108 raft_consensus.cc:385] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.203421 29108 raft_consensus.cc:740] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.203675 29108 consensus_queue.cc:260] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.204380 29108 ts_tablet_manager.cc:1434] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 07:58:45.204847 29108 tablet_bootstrap.cc:492] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:45.205014 28931 tablet_bootstrap.cc:492] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:45.205226 28931 ts_tablet_manager.cc:1403] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.009s
I20260809 07:58:45.206454 28931 raft_consensus.cc:359] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.206737 28931 raft_consensus.cc:385] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.206866 28931 raft_consensus.cc:740] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.207149 29108 tablet_bootstrap.cc:654] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.207129 29085 ts_tablet_manager.cc:1929] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.207281 29085 log.cc:1199] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/03a746a63ed14fb18d0b9f21a32a40ad
I20260809 07:58:45.207180 28931 consensus_queue.cc:260] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.210287 28931 ts_tablet_manager.cc:1434] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 07:58:45.212360 28931 tablet_bootstrap.cc:492] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:45.213996 28931 tablet_bootstrap.cc:654] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.216741 29108 tablet_bootstrap.cc:492] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:45.216917 29108 ts_tablet_manager.cc:1403] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260809 07:58:45.217773 29108 raft_consensus.cc:359] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.217981 29108 raft_consensus.cc:385] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.218072 29108 raft_consensus.cc:740] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.218333 29108 consensus_queue.cc:260] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.218921 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.219301 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.219851 29108 ts_tablet_manager.cc:1434] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:45.226567 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.226925 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.231446 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.055s user 0.006s sys 0.000s Metrics: {"bytes_written":466,"delete_count":0,"lbm_write_time_us":195,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 07:58:45.238456 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.060s user 0.007s sys 0.003s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":12254,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":129,"lbm_read_time_us":437,"lbm_reads_lt_1ms":27,"lbm_write_time_us":337,"lbm_writes_lt_1ms":9,"mutex_wait_us":48543,"peak_mem_usage":32768,"reinsert_count":0,"update_count":3}
I20260809 07:58:45.244961 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":25176}
I20260809 07:58:45.244906 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.000s sys 0.002s Metrics: {"bytes_written":454,"delete_count":0,"lbm_write_time_us":183,"lbm_writes_lt_1ms":4,"mutex_wait_us":19100,"reinsert_count":0,"update_count":2}
I20260809 07:58:45.250713 29003 raft_consensus.cc:493] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.251163 29003 raft_consensus.cc:515] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.251757 29001 consensus_queue.cc:1048] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.251986 29003 leader_election.cc:290] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:45.260982 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1ec621e45e4e57b7fb536f3db739eb" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.261363 28647 raft_consensus.cc:2468] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:45.264228 29029 raft_consensus.cc:493] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.264436 29029 raft_consensus.cc:515] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.265266 29029 leader_election.cc:290] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.268723 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.078s user 0.006s sys 0.000s Metrics: {"bytes_written":466,"delete_count":0,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 07:58:45.271350 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.273319 28740 leader_election.cc:304] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.253218 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1ec621e45e4e57b7fb536f3db739eb" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.264605 29095 tablet_bootstrap.cc:492] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:45.274127 29095 ts_tablet_manager.cc:1403] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.256s user 0.009s sys 0.029s
I20260809 07:58:45.275130 29095 raft_consensus.cc:359] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.275383 29095 raft_consensus.cc:385] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.275470 29095 raft_consensus.cc:740] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.275759 29095 consensus_queue.cc:260] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.276391 29095 ts_tablet_manager.cc:1434] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:45.276788 29095 tablet_bootstrap.cc:492] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:45.266043 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a98f94e5ed406ba4ac318dc3699913" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
W20260809 07:58:45.277828 28565 leader_election.cc:343] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.273274 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a98f94e5ed406ba4ac318dc3699913" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.279249 28826 raft_consensus.cc:2468] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:45.279299 28941 raft_consensus.cc:2804] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.279554 28941 raft_consensus.cc:493] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.279683 28941 raft_consensus.cc:3060] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.279619 29001 consensus_queue.cc:1048] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:58:45.280269 28740 leader_election.cc:343] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.280829 28566 leader_election.cc:304] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:45.281366 29029 raft_consensus.cc:2804] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.281458 29029 raft_consensus.cc:493] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.281570 29029 raft_consensus.cc:3060] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.282979 28941 raft_consensus.cc:515] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.284539 29029 raft_consensus.cc:515] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.286164 29029 leader_election.cc:290] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.287762 28824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a98f94e5ed406ba4ac318dc3699913" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:45.287990 28824 raft_consensus.cc:3060] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.289692 28941 leader_election.cc:290] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:45.290001 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a98f94e5ed406ba4ac318dc3699913" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
W20260809 07:58:45.290619 28565 leader_election.cc:343] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.295755 29029 raft_consensus.cc:493] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.296008 29029 raft_consensus.cc:515] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.296676 29085 ts_tablet_manager.cc:1950] T 03a746a63ed14fb18d0b9f21a32a40ad P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.297576 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7751d1e95a124184848a095dfcc6268e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.297698 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 20) took 1056 ms. Trace:
I20260809 07:58:45.298213 29095 tablet_bootstrap.cc:654] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
W20260809 07:58:45.298671 28565 leader_election.cc:343] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.297883 29085 rpcz_store.cc:276] 0809 07:58:44.240706 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.240776 (+ 70us) service_pool.cc:224] Handling call
0809 07:58:45.002652 (+761876us) ts_tablet_manager.cc:1151] Deleting tablet 03a746a63ed14fb18d0b9f21a32a40ad
0809 07:58:45.002660 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.149970 (+147310us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.207042 (+ 57072us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.297610 (+ 90568us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":761711}
I20260809 07:58:45.299113 29029 leader_election.cc:290] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.299460 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 03a746a63ed14fb18d0b9f21a32a40ad (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.299801 28824 raft_consensus.cc:2468] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:45.301693 28824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7751d1e95a124184848a095dfcc6268e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.301950 28824 raft_consensus.cc:2468] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:45.302330 29085 tablet_replica.cc:333] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.302596 29085 raft_consensus.cc:2243] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:45.303177 29085 raft_consensus.cc:2272] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.304945 29085 ts_tablet_manager.cc:1916] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.305558 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1ec621e45e4e57b7fb536f3db739eb" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:45.305840 28649 raft_consensus.cc:3060] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.309641 28647 raft_consensus.cc:3060] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.313308 28931 tablet_bootstrap.cc:492] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:45.313482 29085 ts_tablet_manager.cc:1929] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.313504 28931 ts_tablet_manager.cc:1403] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.101s user 0.031s sys 0.046s
I20260809 07:58:45.313604 29085 log.cc:1199] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/10733adc026c49399d8d6649c5fe7705
I20260809 07:58:45.313941 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1ec621e45e4e57b7fb536f3db739eb" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:45.314091 29085 ts_tablet_manager.cc:1950] T 10733adc026c49399d8d6649c5fe7705 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.314179 28462 raft_consensus.cc:3060] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.314358 28931 raft_consensus.cc:359] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.314587 28931 raft_consensus.cc:385] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.314679 28931 raft_consensus.cc:740] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.314906 28931 consensus_queue.cc:260] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.315120 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 21) took 1074 ms. Trace:
I20260809 07:58:45.315263 29085 rpcz_store.cc:276] 0809 07:58:44.241082 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.241128 (+ 46us) service_pool.cc:224] Handling call
0809 07:58:45.302223 (+1061095us) ts_tablet_manager.cc:1151] Deleting tablet 10733adc026c49399d8d6649c5fe7705
0809 07:58:45.302234 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.309204 (+ 6970us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.313381 (+ 4177us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.315071 (+ 1690us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1060875}
I20260809 07:58:45.315881 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 10733adc026c49399d8d6649c5fe7705 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.316253 29085 tablet_replica.cc:333] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.316519 29085 raft_consensus.cc:2243] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:45.316865 29085 raft_consensus.cc:2272] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.317378 28462 raft_consensus.cc:2468] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:45.318195 29085 ts_tablet_manager.cc:1916] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.318205 28566 leader_election.cc:304] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:45.318243 29003 consensus_queue.cc:1048] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.318624 28566 leader_election.cc:304] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:45.318890 29003 consensus_queue.cc:1048] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.319036 28961 raft_consensus.cc:2804] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.319186 28961 raft_consensus.cc:697] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:45.319479 28961 consensus_queue.cc:237] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.320502 29036 raft_consensus.cc:2804] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.320626 29036 raft_consensus.cc:493] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.320598 28740 leader_election.cc:304] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.320714 29036 raft_consensus.cc:3060] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.321110 28931 ts_tablet_manager.cc:1434] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260809 07:58:45.321388 29003 raft_consensus.cc:2804] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.321532 29003 raft_consensus.cc:697] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:45.321784 29003 consensus_queue.cc:237] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.324025 29036 raft_consensus.cc:515] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.324707 29036 leader_election.cc:290] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.325546 28931 tablet_bootstrap.cc:492] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:45.326032 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7751d1e95a124184848a095dfcc6268e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
W20260809 07:58:45.326705 28565 leader_election.cc:343] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.327344 28931 tablet_bootstrap.cc:654] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.327828 28649 raft_consensus.cc:2468] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:45.331986 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.105s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":104636}
I20260809 07:58:45.332600 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.060s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":59735}
I20260809 07:58:45.333709 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.107s user 0.006s sys 0.000s Metrics: {"bytes_written":454,"delete_count":0,"lbm_write_time_us":174,"lbm_writes_lt_1ms":4,"mutex_wait_us":66318,"reinsert_count":0,"update_count":2}
I20260809 07:58:45.336921 28824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7751d1e95a124184848a095dfcc6268e" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:45.337149 28824 raft_consensus.cc:3060] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.339028 28649 raft_consensus.cc:3060] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.340906 28824 raft_consensus.cc:2468] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:45.343514 28566 leader_election.cc:304] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:45.344164 29036 raft_consensus.cc:2804] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.344429 29036 raft_consensus.cc:697] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:45.345223 29036 consensus_queue.cc:237] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.347368 29001 consensus_queue.cc:1048] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.348114 29036 raft_consensus.cc:493] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.348333 29036 raft_consensus.cc:515] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.349556 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39664b79fe3246dfbae1cb687bfc8129" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.350340 29085 ts_tablet_manager.cc:1929] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.350545 29036 leader_election.cc:290] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.351152 28824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39664b79fe3246dfbae1cb687bfc8129" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.347862 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.154s user 0.010s sys 0.005s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":12220,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":308,"dirs.run_cpu_time_us":285,"dirs.run_wall_time_us":4869,"lbm_read_time_us":658,"lbm_reads_lt_1ms":27,"lbm_write_time_us":419,"lbm_writes_lt_1ms":9,"mutex_wait_us":36045,"peak_mem_usage":32768,"reinsert_count":0,"update_count":6}
I20260809 07:58:45.351913 28566 leader_election.cc:304] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.352397 29198 raft_consensus.cc:2749] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:58:45.352684 29085 log.cc:1199] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/9bd831b22d6044209063ec4b9a984e6d
I20260809 07:58:45.353634 28931 tablet_bootstrap.cc:492] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:45.353871 28931 ts_tablet_manager.cc:1403] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.006s
I20260809 07:58:45.354954 28931 raft_consensus.cc:359] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.355249 28931 raft_consensus.cc:385] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.355484 28931 raft_consensus.cc:740] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.355921 28931 consensus_queue.cc:260] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.356572 28931 ts_tablet_manager.cc:1434] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 07:58:45.357085 28931 tablet_bootstrap.cc:492] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:45.359447 28931 tablet_bootstrap.cc:654] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.360229 29085 ts_tablet_manager.cc:1950] T 9bd831b22d6044209063ec4b9a984e6d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.363787 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 23) took 1119 ms. Trace:
I20260809 07:58:45.363950 29085 rpcz_store.cc:276] 0809 07:58:44.244173 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.244227 (+ 54us) service_pool.cc:224] Handling call
0809 07:58:45.316170 (+1071943us) ts_tablet_manager.cc:1151] Deleting tablet 9bd831b22d6044209063ec4b9a984e6d
0809 07:58:45.316182 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.329329 (+ 13147us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.350226 (+ 20897us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.363726 (+ 13500us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1071765}
I20260809 07:58:45.364454 29085 tablet_replica.cc:333] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.364574 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 9bd831b22d6044209063ec4b9a984e6d (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.364837 29085 raft_consensus.cc:2243] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:45.365172 29085 raft_consensus.cc:2272] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.373706 29001 raft_consensus.cc:493] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.373879 29001 raft_consensus.cc:515] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.374691 29001 leader_election.cc:290] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:45.377323 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1a97f98f5446daa2523cf10fb8fac4" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.377463 29198 raft_consensus.cc:493] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.377650 29198 raft_consensus.cc:515] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.377974 29085 ts_tablet_manager.cc:1916] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.378477 29198 leader_election.cc:290] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.378733 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1a97f98f5446daa2523cf10fb8fac4" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.378929 28648 raft_consensus.cc:2468] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:45.379822 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1a97f98f5446daa2523cf10fb8fac4" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.379738 29001 consensus_queue.cc:1048] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:58:45.380295 28565 leader_election.cc:343] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:45.383286 28380 consensus_peers.cc:597] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1365599d9764ccfbfb0cb954f83de69. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:45.387547 28740 leader_election.cc:304] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.390787 29001 raft_consensus.cc:2804] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.390906 29001 raft_consensus.cc:493] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.391125 29001 raft_consensus.cc:3060] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.392874 29198 raft_consensus.cc:493] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.393028 29198 raft_consensus.cc:515] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.393714 29198 leader_election.cc:290] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.395391 28648 raft_consensus.cc:3060] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.399569 29014 consensus_queue.cc:1048] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.394326 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c44aace20a640ef8aca975a73af0d93" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.402209 29001 raft_consensus.cc:515] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.402878 29001 leader_election.cc:290] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
W20260809 07:58:45.404465 28740 leader_election.cc:343] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.405364 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1a97f98f5446daa2523cf10fb8fac4" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:45.411208 28149 catalog_manager.cc:5719] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:45.411746 29198 raft_consensus.cc:493] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.411913 29198 raft_consensus.cc:515] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.412537 29198 leader_election.cc:290] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.413378 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff7ca7c344b48b685c829148b184b9b" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
W20260809 07:58:45.413934 28565 leader_election.cc:343] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.416108 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1a97f98f5446daa2523cf10fb8fac4" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:45.416323 28649 raft_consensus.cc:3060] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.420068 28649 raft_consensus.cc:2468] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:45.422881 28149 catalog_manager.cc:5719] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:45.424754 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.425104 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.425436 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.425781 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.426206 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.426543 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.426851 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.428615 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.428669 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.428879 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.429028 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.429117 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.429462 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.429589 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.429697 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.429872 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.429891 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.430099 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.430287 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.430295 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.430653 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.431169 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1a97f98f5446daa2523cf10fb8fac4" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.431272 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c44aace20a640ef8aca975a73af0d93" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.431475 28826 raft_consensus.cc:2393] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24137fd377124538a31e92f3b6df0673 in current term 1: Already voted for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in this term.
I20260809 07:58:45.431483 28825 raft_consensus.cc:2393] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24137fd377124538a31e92f3b6df0673 in current term 1: Already voted for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in this term.
I20260809 07:58:45.431989 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff7ca7c344b48b685c829148b184b9b" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.432261 29014 consensus_queue.cc:1048] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.433846 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.433907 28566 leader_election.cc:304] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.434587 28961 raft_consensus.cc:2749] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:45.435098 28566 leader_election.cc:304] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.435556 28961 raft_consensus.cc:2749] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:45.436033 28740 leader_election.cc:304] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.436362 29003 raft_consensus.cc:2804] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 07:58:45.436430 28740 leader_election.cc:343] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.436544 29003 raft_consensus.cc:697] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
W20260809 07:58:45.436851 28566 leader_election.cc:343] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.436810 29003 consensus_queue.cc:237] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.436973 28566 leader_election.cc:304] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.437618 28961 raft_consensus.cc:2749] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:45.441960 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.442560 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.445201 29014 consensus_queue.cc:1048] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.445811 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":134,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.446312 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.446661 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.446961 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.447487 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.006s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":688,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":2498,"lbm_read_time_us":164,"lbm_reads_lt_1ms":8,"lbm_write_time_us":172,"lbm_writes_lt_1ms":4,"mutex_wait_us":5183,"reinsert_count":0,"update_count":3}
I20260809 07:58:45.447916 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":13727}
I20260809 07:58:45.448164 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.448457 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.448509 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.001s sys 0.003s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":143,"lbm_writes_lt_1ms":4,"mutex_wait_us":12257,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.448875 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.449215 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":6275}
I20260809 07:58:45.449268 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.003s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":688,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1042,"dirs.run_cpu_time_us":235,"dirs.run_wall_time_us":1835,"lbm_read_time_us":166,"lbm_reads_lt_1ms":8,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"mutex_wait_us":4709,"reinsert_count":0,"update_count":3}
I20260809 07:58:45.449753 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.450062 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.450354 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.450203 28149 catalog_manager.cc:5719] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:45.450937 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.451201 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.451318 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.451462 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.455888 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":8663}
I20260809 07:58:45.456168 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":8861}
I20260809 07:58:45.459483 29095 tablet_bootstrap.cc:492] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:45.459664 29095 ts_tablet_manager.cc:1403] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.183s user 0.014s sys 0.021s
I20260809 07:58:45.460608 29095 raft_consensus.cc:359] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.460843 29095 raft_consensus.cc:385] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.460942 29095 raft_consensus.cc:740] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.461205 29095 consensus_queue.cc:260] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.461802 29095 ts_tablet_manager.cc:1434] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 07:58:45.462144 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.462476 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.462728 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.463294 28989 consensus_queue.cc:1048] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.463920 28931 tablet_bootstrap.cc:492] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:45.464082 28931 ts_tablet_manager.cc:1403] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.107s user 0.014s sys 0.032s
I20260809 07:58:45.464924 28931 raft_consensus.cc:359] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.465137 28931 raft_consensus.cc:385] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.465220 28931 raft_consensus.cc:740] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.465530 28931 consensus_queue.cc:260] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.466167 28931 ts_tablet_manager.cc:1434] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 07:58:45.468758 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.469123 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.469770 29095 tablet_bootstrap.cc:492] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:45.470620 29085 ts_tablet_manager.cc:1929] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.470777 29085 log.cc:1199] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/392548585cf1446f99dbc268831cae8e
I20260809 07:58:45.471349 29085 ts_tablet_manager.cc:1950] T 392548585cf1446f99dbc268831cae8e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.474059 29211 consensus_queue.cc:1048] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.475134 29095 tablet_bootstrap.cc:654] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.475723 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":716,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1305,"lbm_read_time_us":224,"lbm_reads_lt_1ms":12,"lbm_write_time_us":447,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":4}
I20260809 07:58:45.476310 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.480816 29211 consensus_queue.cc:1048] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.481065 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":17857}
I20260809 07:58:45.481492 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":18219}
I20260809 07:58:45.482089 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":177,"lbm_writes_lt_1ms":4,"mutex_wait_us":1237,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.482487 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.003s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":716,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":157,"lbm_reads_lt_1ms":8,"lbm_write_time_us":155,"lbm_writes_lt_1ms":4,"mutex_wait_us":7386,"reinsert_count":0,"update_count":4}
I20260809 07:58:45.485327 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.485668 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.488200 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.004s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":351,"lbm_writes_lt_1ms":4,"mutex_wait_us":12354,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.471720 28149 catalog_manager.cc:5719] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:45.499464 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.499761 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.500022 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.505375 28942 raft_consensus.cc:493] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2a091b20dbc4ed2b3bf6c960df964c5)
I20260809 07:58:45.505575 28942 raft_consensus.cc:515] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.506220 28942 leader_election.cc:290] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.507130 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":159,"lbm_writes_lt_1ms":4,"mutex_wait_us":32066,"reinsert_count":0,"update_count":1}
W20260809 07:58:45.507406 28380 leader_election.cc:343] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Not found: Tablet not found: 9fec26011dec4e91868a1cbe71cdf371
I20260809 07:58:45.508268 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.104666
I20260809 07:58:45.509444 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"mutex_wait_us":13489,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.510043 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.510466 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
W20260809 07:58:45.513445 28382 leader_election.cc:343] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Not found: Tablet not found: 9fec26011dec4e91868a1cbe71cdf371
I20260809 07:58:45.513577 28382 leader_election.cc:304] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.514017 29014 raft_consensus.cc:2749] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:58:45.522639 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 22) took 1278 ms. Trace:
I20260809 07:58:45.522845 29085 rpcz_store.cc:276] 0809 07:58:44.244061 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.244595 (+ 534us) service_pool.cc:224] Handling call
0809 07:58:45.364349 (+1119754us) ts_tablet_manager.cc:1151] Deleting tablet 392548585cf1446f99dbc268831cae8e
0809 07:58:45.364361 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.423136 (+ 58775us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.470501 (+ 47365us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.522593 (+ 52092us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1119563}
I20260809 07:58:45.524853 29085 tablet_replica.cc:333] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.524994 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 392548585cf1446f99dbc268831cae8e (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.525235 29085 raft_consensus.cc:2243] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:45.525861 29085 raft_consensus.cc:2272] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.526964 29085 ts_tablet_manager.cc:1916] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.537436 28941 consensus_queue.cc:1048] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.540024 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":39340}
I20260809 07:58:45.540501 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":37178}
I20260809 07:58:45.540966 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.001s sys 0.003s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":149,"lbm_writes_lt_1ms":4,"mutex_wait_us":34766,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.541554 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.065s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":746,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":8100,"dirs.run_cpu_time_us":125,"dirs.run_wall_time_us":2403,"lbm_read_time_us":195,"lbm_reads_lt_1ms":8,"lbm_write_time_us":204,"lbm_writes_lt_1ms":4,"mutex_wait_us":19307,"reinsert_count":0,"update_count":5}
I20260809 07:58:45.542194 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.542518 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.542788 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.543025 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.545630 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.545894 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.546128 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.551329 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":783,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":53,"lbm_read_time_us":228,"lbm_reads_lt_1ms":12,"lbm_write_time_us":166,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":6}
I20260809 07:58:45.558092 29095 tablet_bootstrap.cc:492] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:45.559293 29095 ts_tablet_manager.cc:1403] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.090s user 0.017s sys 0.054s
I20260809 07:58:45.560293 29095 raft_consensus.cc:359] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.560600 29095 raft_consensus.cc:385] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.560750 29095 raft_consensus.cc:740] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.561110 29095 consensus_queue.cc:260] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.562249 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.054s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":6,"cfile_cache_miss_bytes":1081,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":66,"lbm_read_time_us":322,"lbm_reads_lt_1ms":14,"lbm_write_time_us":179,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":6}
I20260809 07:58:45.563386 29095 ts_tablet_manager.cc:1434] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:45.563741 29095 tablet_bootstrap.cc:492] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:45.565495 29085 ts_tablet_manager.cc:1929] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.565672 29095 tablet_bootstrap.cc:654] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.565686 29085 log.cc:1199] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/b95825d687ab41878ef822421d853385
I20260809 07:58:45.569204 29085 ts_tablet_manager.cc:1950] T b95825d687ab41878ef822421d853385 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.570003 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 25) took 1324 ms. Trace:
I20260809 07:58:45.570118 29085 rpcz_store.cc:276] 0809 07:58:44.246001 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.246273 (+ 272us) service_pool.cc:224] Handling call
0809 07:58:45.524781 (+1278508us) ts_tablet_manager.cc:1151] Deleting tablet b95825d687ab41878ef822421d853385
0809 07:58:45.524791 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.562052 (+ 37261us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.565410 (+ 3358us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.569944 (+ 4534us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1278308}
I20260809 07:58:45.570568 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet b95825d687ab41878ef822421d853385 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.571838 29085 tablet_replica.cc:333] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.572042 29085 raft_consensus.cc:2243] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:45.572252 29085 raft_consensus.cc:2272] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.572882 29085 ts_tablet_manager.cc:1916] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.579519 28941 raft_consensus.cc:493] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.579707 28941 raft_consensus.cc:515] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.580305 28941 leader_election.cc:290] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:45.581043 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48541470e51457d980a8c3ea27b060f" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.581305 28648 raft_consensus.cc:2468] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:45.581722 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48541470e51457d980a8c3ea27b060f" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.582129 28941 consensus_queue.cc:1048] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.582772 29085 ts_tablet_manager.cc:1929] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.582919 29085 log.cc:1199] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/9fec26011dec4e91868a1cbe71cdf371
I20260809 07:58:45.583189 29095 tablet_bootstrap.cc:492] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:45.583331 29095 ts_tablet_manager.cc:1403] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.007s
I20260809 07:58:45.583386 28740 leader_election.cc:304] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.583662 28941 raft_consensus.cc:2804] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.583717 29085 ts_tablet_manager.cc:1950] T 9fec26011dec4e91868a1cbe71cdf371 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.583791 28941 raft_consensus.cc:493] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.583904 28941 raft_consensus.cc:3060] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.584157 29095 raft_consensus.cc:359] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.584353 29095 raft_consensus.cc:385] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.584460 29095 raft_consensus.cc:740] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.584581 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 24) took 1338 ms. Trace:
I20260809 07:58:45.584756 29085 rpcz_store.cc:276] 0809 07:58:44.245863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.246949 (+ 1086us) service_pool.cc:224] Handling call
0809 07:58:45.571752 (+1324803us) ts_tablet_manager.cc:1151] Deleting tablet 9fec26011dec4e91868a1cbe71cdf371
0809 07:58:45.571761 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.576542 (+ 4781us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.582673 (+ 6131us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.584540 (+ 1867us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1324596}
I20260809 07:58:45.585151 29085 tablet_replica.cc:333] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.585379 29085 raft_consensus.cc:2243] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:45.585731 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 9fec26011dec4e91868a1cbe71cdf371 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.585783 29085 raft_consensus.cc:2272] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.586076 28961 raft_consensus.cc:493] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.586282 28961 raft_consensus.cc:515] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.586747 29085 ts_tablet_manager.cc:1916] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.587082 28961 leader_election.cc:290] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.588321 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550838d288be4ecf8a9dd10aeda6467a" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
W20260809 07:58:45.589284 28565 leader_election.cc:343] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.590749 28941 raft_consensus.cc:515] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.584765 29095 consensus_queue.cc:260] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.592504 29095 ts_tablet_manager.cc:1434] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.009s user 0.001s sys 0.002s
I20260809 07:58:45.592839 29095 tablet_bootstrap.cc:492] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:45.593648 29085 ts_tablet_manager.cc:1929] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.593830 29085 log.cc:1199] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/437e82d85fc445ce8ae904c2c71d08c7
I20260809 07:58:45.594393 29085 ts_tablet_manager.cc:1950] T 437e82d85fc445ce8ae904c2c71d08c7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.595047 29095 tablet_bootstrap.cc:654] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.595209 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 27) took 1348 ms. Trace:
I20260809 07:58:45.595350 29085 rpcz_store.cc:276] 0809 07:58:44.246219 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.247094 (+ 875us) service_pool.cc:224] Handling call
0809 07:58:45.585086 (+1337992us) ts_tablet_manager.cc:1151] Deleting tablet 437e82d85fc445ce8ae904c2c71d08c7
0809 07:58:45.585094 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.590117 (+ 5023us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.593577 (+ 3460us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.595132 (+ 1555us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1337758}
I20260809 07:58:45.595746 29085 tablet_replica.cc:333] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.595860 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 437e82d85fc445ce8ae904c2c71d08c7 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.595948 29085 raft_consensus.cc:2243] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:45.596146 29085 raft_consensus.cc:2272] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.596764 29085 ts_tablet_manager.cc:1916] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.601220 28941 leader_election.cc:290] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
W20260809 07:58:45.601433 28740 leader_election.cc:343] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.602130 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48541470e51457d980a8c3ea27b060f" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:45.603003 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48541470e51457d980a8c3ea27b060f" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:45.603173 29085 ts_tablet_manager.cc:1929] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.603214 28648 raft_consensus.cc:3060] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.603304 29085 log.cc:1199] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/841c52b2284042a8a4dab34f8ed13fe2
I20260809 07:58:45.604003 29085 ts_tablet_manager.cc:1950] T 841c52b2284042a8a4dab34f8ed13fe2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.604574 29095 tablet_bootstrap.cc:492] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:45.604743 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 26) took 1358 ms. Trace:
I20260809 07:58:45.604897 29085 rpcz_store.cc:276] 0809 07:58:44.246110 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.248296 (+ 2186us) service_pool.cc:224] Handling call
0809 07:58:45.595648 (+1347352us) ts_tablet_manager.cc:1151] Deleting tablet 841c52b2284042a8a4dab34f8ed13fe2
0809 07:58:45.595659 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.599935 (+ 4276us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.603080 (+ 3145us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.604694 (+ 1614us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1347152}
I20260809 07:58:45.605376 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 841c52b2284042a8a4dab34f8ed13fe2 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.605715 29085 tablet_replica.cc:333] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.605918 29085 raft_consensus.cc:2243] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:45.606087 29085 raft_consensus.cc:2272] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.606289 28648 raft_consensus.cc:2468] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:45.606431 28961 consensus_queue.cc:1048] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.606709 29085 ts_tablet_manager.cc:1916] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.604746 29095 ts_tablet_manager.cc:1403] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260809 07:58:45.607757 29095 raft_consensus.cc:359] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.608567 29095 raft_consensus.cc:385] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.608767 29095 raft_consensus.cc:740] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.609025 29095 consensus_queue.cc:260] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.609579 29095 ts_tablet_manager.cc:1434] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:45.610004 29095 tablet_bootstrap.cc:492] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
W20260809 07:58:45.610401 28740 leader_election.cc:343] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.610517 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550838d288be4ecf8a9dd10aeda6467a" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.610831 28826 raft_consensus.cc:2468] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:45.610895 28740 leader_election.cc:304] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:45.611202 28941 raft_consensus.cc:2804] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.611878 29095 tablet_bootstrap.cc:654] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.611915 28941 raft_consensus.cc:697] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:45.612298 28941 consensus_queue.cc:237] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.614020 28566 leader_election.cc:304] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:45.614575 28961 raft_consensus.cc:2804] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.614722 28961 raft_consensus.cc:493] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.614871 28961 raft_consensus.cc:3060] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.615247 29085 ts_tablet_manager.cc:1929] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.615410 29085 log.cc:1199] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/a2d60a37346b469781732b692cf55c63
I20260809 07:58:45.616043 29085 ts_tablet_manager.cc:1950] T a2d60a37346b469781732b692cf55c63 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.616976 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 29) took 1365 ms. Trace:
I20260809 07:58:45.617107 29085 rpcz_store.cc:276] 0809 07:58:44.251322 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.251623 (+ 301us) service_pool.cc:224] Handling call
0809 07:58:45.605652 (+1354029us) ts_tablet_manager.cc:1151] Deleting tablet a2d60a37346b469781732b692cf55c63
0809 07:58:45.605661 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.610602 (+ 4941us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.615148 (+ 4546us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.616915 (+ 1767us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1353839}
I20260809 07:58:45.617690 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet a2d60a37346b469781732b692cf55c63 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.618856 28961 raft_consensus.cc:515] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.620023 28961 leader_election.cc:290] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.620491 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550838d288be4ecf8a9dd10aeda6467a" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:45.621080 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550838d288be4ecf8a9dd10aeda6467a" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:45.621389 28826 raft_consensus.cc:3060] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.621464 29085 tablet_replica.cc:333] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.621667 29085 raft_consensus.cc:2243] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:45.622037 29085 raft_consensus.cc:2272] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.622645 29085 ts_tablet_manager.cc:1916] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.624516 28826 raft_consensus.cc:2468] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:45.625274 28566 leader_election.cc:304] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.625710 28961 raft_consensus.cc:2804] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.625897 28961 raft_consensus.cc:697] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
W20260809 07:58:45.626536 28565 leader_election.cc:343] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:45.626302 28961 consensus_queue.cc:237] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.632143 28153 catalog_manager.cc:5719] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:45.633165 29085 ts_tablet_manager.cc:1929] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.633383 29085 log.cc:1199] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/5745cdf610dd4552bfd339a7ed04e95c
I20260809 07:58:45.633955 29085 ts_tablet_manager.cc:1950] T 5745cdf610dd4552bfd339a7ed04e95c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.634300 29198 consensus_queue.cc:1048] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.635035 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 28) took 1383 ms. Trace:
I20260809 07:58:45.635185 29085 rpcz_store.cc:276] 0809 07:58:44.251162 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.251846 (+ 684us) service_pool.cc:224] Handling call
0809 07:58:45.621299 (+1369453us) ts_tablet_manager.cc:1151] Deleting tablet 5745cdf610dd4552bfd339a7ed04e95c
0809 07:58:45.621309 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.628091 (+ 6782us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.632988 (+ 4897us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.634984 (+ 1996us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1369279}
I20260809 07:58:45.635618 29085 tablet_replica.cc:333] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.635951 29085 raft_consensus.cc:2243] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:45.636305 29085 raft_consensus.cc:2272] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.637151 29085 ts_tablet_manager.cc:1916] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.637531 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 5745cdf610dd4552bfd339a7ed04e95c (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.637712 29095 tablet_bootstrap.cc:492] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:45.637876 29095 ts_tablet_manager.cc:1403] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.011s
I20260809 07:58:45.638752 29095 raft_consensus.cc:359] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.638949 29095 raft_consensus.cc:385] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.639045 29095 raft_consensus.cc:740] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.639331 29095 consensus_queue.cc:260] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.639909 29095 ts_tablet_manager.cc:1434] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:45.640275 29095 tablet_bootstrap.cc:492] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:45.642203 29095 tablet_bootstrap.cc:654] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:45.645399 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.645740 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.646039 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.646298 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.646534 29085 ts_tablet_manager.cc:1929] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.646735 29085 log.cc:1199] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/b1365599d9764ccfbfb0cb954f83de69
I20260809 07:58:45.647297 29085 ts_tablet_manager.cc:1950] T b1365599d9764ccfbfb0cb954f83de69 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.647827 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.648151 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.648242 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.648384 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 31) took 1396 ms. Trace:
I20260809 07:58:45.648447 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.648521 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.648514 29085 rpcz_store.cc:276] 0809 07:58:44.251576 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.252045 (+ 469us) service_pool.cc:224] Handling call
0809 07:58:45.635543 (+1383498us) ts_tablet_manager.cc:1151] Deleting tablet b1365599d9764ccfbfb0cb954f83de69
0809 07:58:45.635552 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.642154 (+ 6602us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.646427 (+ 4273us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.648306 (+ 1879us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1383326}
I20260809 07:58:45.648759 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.648749 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.649001 29085 tablet_replica.cc:333] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:45.649194 29085 raft_consensus.cc:2243] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:45.649257 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101036
I20260809 07:58:45.649489 29085 raft_consensus.cc:2272] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:45.649583 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101036
I20260809 07:58:45.649863 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101036
I20260809 07:58:45.650233 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.650321 29085 ts_tablet_manager.cc:1916] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:45.650491 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.650892 29014 raft_consensus.cc:493] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.651089 29014 raft_consensus.cc:515] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.651806 29014 leader_election.cc:290] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.653484 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet b1365599d9764ccfbfb0cb954f83de69 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.658844 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.659217 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.659507 29085 ts_tablet_manager.cc:1929] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:45.659688 29085 log.cc:1199] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/6b81ec52183e4bd28ed4d17ed15f5414
I20260809 07:58:45.660250 29085 ts_tablet_manager.cc:1950] T 6b81ec52183e4bd28ed4d17ed15f5414 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:45.661327 29085 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 30) took 1409 ms. Trace:
I20260809 07:58:45.661540 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48541470e51457d980a8c3ea27b060f" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.661512 29085 rpcz_store.cc:276] 0809 07:58:44.251466 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:44.252339 (+ 873us) service_pool.cc:224] Handling call
0809 07:58:45.648894 (+1396555us) ts_tablet_manager.cc:1151] Deleting tablet 6b81ec52183e4bd28ed4d17ed15f5414
0809 07:58:45.648907 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:45.654955 (+ 6048us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.659383 (+ 4428us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:45.661258 (+ 1875us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1396371}
I20260809 07:58:45.661815 28648 raft_consensus.cc:2393] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in current term 1: Already voted for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in this term.
I20260809 07:58:45.662045 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":161,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.662822 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.663196 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.662516 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 6b81ec52183e4bd28ed4d17ed15f5414 (table table-0 [id=708dcc135eac410bbf10b0db8a172f84]) successfully deleted
I20260809 07:58:45.663493 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.663691 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101036
I20260809 07:58:45.665611 28149 catalog_manager.cc:5719] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:45.666318 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101036
I20260809 07:58:45.666604 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.667357 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.003s sys 0.002s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":175,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.667816 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.668097 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.671578 29095 tablet_bootstrap.cc:492] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:45.671772 29095 ts_tablet_manager.cc:1403] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.019s
I20260809 07:58:45.672762 29095 raft_consensus.cc:359] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.672993 29095 raft_consensus.cc:385] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:45.673080 29095 raft_consensus.cc:740] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:45.673452 29095 consensus_queue.cc:260] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.673789 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.011s sys 0.001s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":12472,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":138,"lbm_read_time_us":477,"lbm_reads_lt_1ms":27,"lbm_write_time_us":376,"lbm_writes_lt_1ms":9,"mutex_wait_us":4064,"peak_mem_usage":32768,"reinsert_count":0,"update_count":7}
I20260809 07:58:45.674014 29095 ts_tablet_manager.cc:1434] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:45.674449 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.674820 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.675123 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.675387 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.675554 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48541470e51457d980a8c3ea27b060f" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.676474 28382 leader_election.cc:304] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.677678 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.677918 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.678159 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.678586 29014 raft_consensus.cc:3060] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.680553 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.680910 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.681308 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.681386 29014 raft_consensus.cc:2749] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:45.684422 29014 consensus_queue.cc:1048] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.684618 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.002s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.687872 29014 raft_consensus.cc:493] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.688051 29014 raft_consensus.cc:515] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.689145 29014 leader_election.cc:290] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.689146 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1a97f98f5446daa2523cf10fb8fac4" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.689414 28649 raft_consensus.cc:2393] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in current term 1: Already voted for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in this term.
I20260809 07:58:45.689896 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1a97f98f5446daa2523cf10fb8fac4" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.690433 28382 leader_election.cc:304] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.690773 28989 raft_consensus.cc:3060] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.692945 28989 raft_consensus.cc:2749] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:45.695327 28942 consensus_queue.cc:1048] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.696915 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":130,"lbm_writes_lt_1ms":4,"mutex_wait_us":10705,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.697103 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":15337}
I20260809 07:58:45.697388 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":15228}
I20260809 07:58:45.693440 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.013s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":12472,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":165,"dirs.run_cpu_time_us":302,"dirs.run_wall_time_us":1425,"lbm_read_time_us":586,"lbm_reads_lt_1ms":27,"lbm_write_time_us":352,"lbm_writes_lt_1ms":9,"mutex_wait_us":27,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":13123456,"update_count":7}
I20260809 07:58:45.706115 29036 consensus_queue.cc:1048] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.711418 28462 raft_consensus.cc:3060] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.714993 29036 consensus_queue.cc:1048] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.729990 29036 consensus_queue.cc:1048] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.734181 28462 raft_consensus.cc:3060] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.737123 28941 raft_consensus.cc:493] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.737316 28941 raft_consensus.cc:515] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.738018 28941 leader_election.cc:290] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:45.738396 29014 raft_consensus.cc:493] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:45.738495 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff7ca7c344b48b685c829148b184b9b" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.738548 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff7ca7c344b48b685c829148b184b9b" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:45.738554 29014 raft_consensus.cc:515] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.738740 28648 raft_consensus.cc:2468] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:45.738768 28464 raft_consensus.cc:2468] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:45.739243 28740 leader_election.cc:304] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.739315 28961 consensus_queue.cc:1048] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.739477 29014 leader_election.cc:290] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:45.739796 28941 raft_consensus.cc:2804] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:45.739902 28941 raft_consensus.cc:493] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:45.739926 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7751d1e95a124184848a095dfcc6268e" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:45.740025 28941 raft_consensus.cc:3060] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.740586 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7751d1e95a124184848a095dfcc6268e" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:45.741235 28382 leader_election.cc:304] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:45.741801 29014 raft_consensus.cc:2749] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:58:45.763392 28941 raft_consensus.cc:515] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.764021 28941 leader_election.cc:290] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:45.764422 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff7ca7c344b48b685c829148b184b9b" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:45.764535 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff7ca7c344b48b685c829148b184b9b" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:45.764632 28464 raft_consensus.cc:3060] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.764751 28648 raft_consensus.cc:3060] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:45.767170 28464 raft_consensus.cc:2468] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:45.767221 28648 raft_consensus.cc:2468] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:45.767629 28740 leader_election.cc:304] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:45.767921 28941 raft_consensus.cc:2804] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:45.768119 28941 raft_consensus.cc:697] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:45.768422 28941 consensus_queue.cc:237] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:45.771306 28153 catalog_manager.cc:5719] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:45.777541 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.777875 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.778244 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.778692 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.778976 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.779335 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.781932 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.003s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.782601 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.782877 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.783138 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.783452 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:45.783880 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.783852 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.784987 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:45.785228 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:45.789261 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.003s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":177,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:45.789279 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.004s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":670,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":71,"lbm_read_time_us":100,"lbm_reads_lt_1ms":8,"lbm_write_time_us":151,"lbm_writes_lt_1ms":4,"mutex_wait_us":2660,"reinsert_count":0,"update_count":2}
I20260809 07:58:45.793740 28941 consensus_queue.cc:1048] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.795522 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.003s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":670,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":76,"lbm_read_time_us":176,"lbm_reads_lt_1ms":8,"lbm_write_time_us":125,"lbm_writes_lt_1ms":4,"mutex_wait_us":3776,"reinsert_count":0,"update_count":2}
I20260809 07:58:45.800002 29212 consensus_queue.cc:1048] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.867151 29212 consensus_queue.cc:1048] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:45.873153 29212 consensus_queue.cc:1048] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.026383 28961 consensus_queue.cc:1048] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.033257 28464 raft_consensus.cc:3060] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.036193 28961 consensus_queue.cc:1048] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.069932 29211 consensus_queue.cc:1048] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.075753 28941 consensus_queue.cc:1048] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.100673 28153 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51156:
table { table_name: "table-1" } modify_external_catalogs: true
I20260809 07:58:46.100832 28153 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51156:
table { table_name: "table-1" } modify_external_catalogs: true
I20260809 07:58:46.114172 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.114545 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.114845 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.115170 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.115370 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 6265b1b590cc40739365212572bfc6ed
I20260809 07:58:46.115700 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.608293
I20260809 07:58:46.116673 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 6265b1b590cc40739365212572bfc6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.116840 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 193f851a4c1043e1af5aaf4b7190730d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.116982 29085 tablet_replica.cc:333] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.117234 29085 raft_consensus.cc:2243] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.117511 29085 raft_consensus.cc:2272] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.117511 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 6265b1b590cc40739365212572bfc6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.117866 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 6265b1b590cc40739365212572bfc6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.118577 29085 ts_tablet_manager.cc:1916] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.120716 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.002s sys 0.000s Metrics: {"bytes_written":917,"delete_count":0,"lbm_write_time_us":180,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260809 07:58:46.120780 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":5864}
I20260809 07:58:46.120997 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.608293
I20260809 07:58:46.121322 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.121435 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.121680 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.121985 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.122254 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.122527 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.122833 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.123075 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.123457 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.608293
I20260809 07:58:46.125403 29085 ts_tablet_manager.cc:1929] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.125643 29085 log.cc:1199] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/6265b1b590cc40739365212572bfc6ed
I20260809 07:58:46.125697 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.126117 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.126523 29085 ts_tablet_manager.cc:1950] T 6265b1b590cc40739365212572bfc6ed P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.127588 29085 tablet_replica.cc:333] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.127837 29085 raft_consensus.cc:2243] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.128281 29085 raft_consensus.cc:2272] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.129298 29085 ts_tablet_manager.cc:1916] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.133872 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 6265b1b590cc40739365212572bfc6ed (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.134584 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e
I20260809 07:58:46.134466 29257 tablet_replica.cc:333] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.135262 29257 raft_consensus.cc:2243] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.135977 29256 tablet_replica.cc:333] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.136343 29256 raft_consensus.cc:2243] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.136667 29256 raft_consensus.cc:2272] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.137409 29085 ts_tablet_manager.cc:1929] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.137552 29085 log.cc:1199] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/193f851a4c1043e1af5aaf4b7190730d
I20260809 07:58:46.137670 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.137755 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.137977 29256 ts_tablet_manager.cc:1916] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.138054 29085 ts_tablet_manager.cc:1950] T 193f851a4c1043e1af5aaf4b7190730d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.138242 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.004s sys 0.001s Metrics: {"bytes_written":917,"delete_count":0,"lbm_write_time_us":196,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260809 07:58:46.135766 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.139251 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 193f851a4c1043e1af5aaf4b7190730d (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.139478 29257 raft_consensus.cc:2272] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.139812 29085 tablet_replica.cc:333] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.139817 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 75642018d47b45e6ba74466c6d5ee95e
I20260809 07:58:46.140039 29085 raft_consensus.cc:2243] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.140213 29085 raft_consensus.cc:2272] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.140926 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 75642018d47b45e6ba74466c6d5ee95e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.141100 29085 ts_tablet_manager.cc:1916] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.141508 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 75642018d47b45e6ba74466c6d5ee95e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.141752 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 3c44aace20a640ef8aca975a73af0d93
I20260809 07:58:46.141753 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 75642018d47b45e6ba74466c6d5ee95e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.141765 29257 ts_tablet_manager.cc:1916] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.142486 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 3c44aace20a640ef8aca975a73af0d93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.144107 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 3c44aace20a640ef8aca975a73af0d93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.144529 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 39664b79fe3246dfbae1cb687bfc8129
I20260809 07:58:46.144531 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 3c44aace20a640ef8aca975a73af0d93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.145179 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 39664b79fe3246dfbae1cb687bfc8129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.146433 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.012s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":13105,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":87,"lbm_read_time_us":487,"lbm_reads_lt_1ms":27,"lbm_write_time_us":336,"lbm_writes_lt_1ms":9,"mutex_wait_us":4614,"peak_mem_usage":32768,"reinsert_count":0,"update_count":19}
I20260809 07:58:46.147792 29085 ts_tablet_manager.cc:1929] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.147977 29085 log.cc:1199] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/15d0b5e3e19e44f0a2414e7ce3da6d7e
I20260809 07:58:46.148409 29085 ts_tablet_manager.cc:1950] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.148844 29257 ts_tablet_manager.cc:1929] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.148633 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":13105,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":91,"lbm_read_time_us":634,"lbm_reads_lt_1ms":27,"lbm_write_time_us":422,"lbm_writes_lt_1ms":9,"mutex_wait_us":4833,"peak_mem_usage":32768,"reinsert_count":0,"update_count":19}
I20260809 07:58:46.148979 29257 log.cc:1199] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/6265b1b590cc40739365212572bfc6ed
I20260809 07:58:46.149521 29085 tablet_replica.cc:333] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.149559 29257 ts_tablet_manager.cc:1950] T 6265b1b590cc40739365212572bfc6ed P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.149729 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 39664b79fe3246dfbae1cb687bfc8129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.149755 29085 raft_consensus.cc:2243] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.149804 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.149966 29085 raft_consensus.cc:2272] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.150699 29085 ts_tablet_manager.cc:1916] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.151153 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 6265b1b590cc40739365212572bfc6ed (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.151456 29257 tablet_replica.cc:333] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.151675 29257 raft_consensus.cc:2243] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.152084 29257 raft_consensus.cc:2272] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.153313 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 33198e831822470db98234b54d09c6bc
I20260809 07:58:46.153790 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 39664b79fe3246dfbae1cb687bfc8129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.154117 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 33198e831822470db98234b54d09c6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.154222 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet f47e81a4dd5943f7873144e373bab017
I20260809 07:58:46.154685 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 33198e831822470db98234b54d09c6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.154912 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet f47e81a4dd5943f7873144e373bab017 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.155489 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet f47e81a4dd5943f7873144e373bab017 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.155663 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet f47e81a4dd5943f7873144e373bab017 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.155694 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 193f851a4c1043e1af5aaf4b7190730d
I20260809 07:58:46.155792 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 33198e831822470db98234b54d09c6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.156301 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 193f851a4c1043e1af5aaf4b7190730d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.156661 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 193f851a4c1043e1af5aaf4b7190730d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.157070 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 7751d1e95a124184848a095dfcc6268e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.157117 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 193f851a4c1043e1af5aaf4b7190730d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.157315 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 12a98f94e5ed406ba4ac318dc3699913
I20260809 07:58:46.157749 29257 ts_tablet_manager.cc:1916] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.157998 29085 ts_tablet_manager.cc:1929] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.158139 29085 log.cc:1199] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/75642018d47b45e6ba74466c6d5ee95e
I20260809 07:58:46.158241 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 12a98f94e5ed406ba4ac318dc3699913 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.158375 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 12a98f94e5ed406ba4ac318dc3699913 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.158794 29085 ts_tablet_manager.cc:1950] T 75642018d47b45e6ba74466c6d5ee95e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.158926 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 12a98f94e5ed406ba4ac318dc3699913 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.159761 29085 tablet_replica.cc:333] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.159888 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 75642018d47b45e6ba74466c6d5ee95e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.159974 29085 raft_consensus.cc:2243] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.160269 29085 raft_consensus.cc:2272] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.160975 29085 ts_tablet_manager.cc:1916] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.161468 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 2a54a36e22474698a198d8c25bafa16b
I20260809 07:58:46.161628 29256 ts_tablet_manager.cc:1929] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.161803 29256 log.cc:1199] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/6265b1b590cc40739365212572bfc6ed
I20260809 07:58:46.162062 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a54a36e22474698a198d8c25bafa16b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.162365 29256 ts_tablet_manager.cc:1950] T 6265b1b590cc40739365212572bfc6ed P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.162840 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a54a36e22474698a198d8c25bafa16b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.163337 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 7751d1e95a124184848a095dfcc6268e
I20260809 07:58:46.163785 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a54a36e22474698a198d8c25bafa16b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.163899 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 7751d1e95a124184848a095dfcc6268e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.164211 29256 tablet_replica.cc:333] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.164315 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 6265b1b590cc40739365212572bfc6ed (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.164479 29256 raft_consensus.cc:2243] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.164669 29256 raft_consensus.cc:2272] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.165033 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 7751d1e95a124184848a095dfcc6268e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.165439 29257 ts_tablet_manager.cc:1929] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.165539 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet fc1ec621e45e4e57b7fb536f3db739eb
I20260809 07:58:46.165714 29257 log.cc:1199] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/15d0b5e3e19e44f0a2414e7ce3da6d7e
I20260809 07:58:46.165928 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 7751d1e95a124184848a095dfcc6268e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.166178 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet fc1ec621e45e4e57b7fb536f3db739eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.166230 29257 ts_tablet_manager.cc:1950] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.166405 29256 ts_tablet_manager.cc:1916] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.166987 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet fc1ec621e45e4e57b7fb536f3db739eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.167266 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.167367 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet ab1a97f98f5446daa2523cf10fb8fac4
I20260809 07:58:46.167544 29085 ts_tablet_manager.cc:1929] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.167668 29085 log.cc:1199] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/3c44aace20a640ef8aca975a73af0d93
I20260809 07:58:46.167915 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet fc1ec621e45e4e57b7fb536f3db739eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.168171 29085 ts_tablet_manager.cc:1950] T 3c44aace20a640ef8aca975a73af0d93 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.168311 29257 tablet_replica.cc:333] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.168352 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet ab1a97f98f5446daa2523cf10fb8fac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.168612 29257 raft_consensus.cc:2243] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.169000 29257 raft_consensus.cc:2272] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.169128 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet ab1a97f98f5446daa2523cf10fb8fac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.169237 29085 tablet_replica.cc:333] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.169436 29085 raft_consensus.cc:2243] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.169648 29085 raft_consensus.cc:2272] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.169504 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 3c44aace20a640ef8aca975a73af0d93 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.169970 29257 ts_tablet_manager.cc:1916] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.170244 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 550838d288be4ecf8a9dd10aeda6467a
I20260809 07:58:46.170686 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet ab1a97f98f5446daa2523cf10fb8fac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.170567 29085 ts_tablet_manager.cc:1916] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.170864 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 550838d288be4ecf8a9dd10aeda6467a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.171465 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 550838d288be4ecf8a9dd10aeda6467a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.172104 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet a48541470e51457d980a8c3ea27b060f
I20260809 07:58:46.172235 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 550838d288be4ecf8a9dd10aeda6467a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.172768 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet a48541470e51457d980a8c3ea27b060f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.173241 29256 ts_tablet_manager.cc:1929] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.173353 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet a48541470e51457d980a8c3ea27b060f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.173480 29256 log.cc:1199] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/15d0b5e3e19e44f0a2414e7ce3da6d7e
I20260809 07:58:46.173995 29256 ts_tablet_manager.cc:1950] T 15d0b5e3e19e44f0a2414e7ce3da6d7e P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.174218 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet a48541470e51457d980a8c3ea27b060f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.173965 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet eff7ca7c344b48b685c829148b184b9b
I20260809 07:58:46.175037 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet eff7ca7c344b48b685c829148b184b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:46.175379 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 15d0b5e3e19e44f0a2414e7ce3da6d7e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.175525 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet eff7ca7c344b48b685c829148b184b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:46.175846 29256 tablet_replica.cc:333] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.176081 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet eff7ca7c344b48b685c829148b184b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:46 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:46.176366 29256 raft_consensus.cc:2243] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.176612 29256 raft_consensus.cc:2272] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.177261 29085 ts_tablet_manager.cc:1929] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.177416 29085 log.cc:1199] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/39664b79fe3246dfbae1cb687bfc8129
I20260809 07:58:46.177644 29256 ts_tablet_manager.cc:1916] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.177572 28153 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51156:
name: "table-2"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
I20260809 07:58:46.177866 29085 ts_tablet_manager.cc:1950] T 39664b79fe3246dfbae1cb687bfc8129 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
W20260809 07:58:46.178753 28153 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:58:46.178823 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 39664b79fe3246dfbae1cb687bfc8129 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.179126 29085 tablet_replica.cc:333] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.179343 29085 raft_consensus.cc:2243] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.179522 29085 raft_consensus.cc:2272] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.179661 29257 ts_tablet_manager.cc:1929] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.179816 29257 log.cc:1199] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/75642018d47b45e6ba74466c6d5ee95e
I20260809 07:58:46.180342 29257 ts_tablet_manager.cc:1950] T 75642018d47b45e6ba74466c6d5ee95e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.180537 29085 ts_tablet_manager.cc:1916] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.181399 29257 tablet_replica.cc:333] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.181456 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 75642018d47b45e6ba74466c6d5ee95e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.181661 29257 raft_consensus.cc:2243] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.181902 29257 raft_consensus.cc:2272] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.182775 29257 ts_tablet_manager.cc:1916] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.185698 29256 ts_tablet_manager.cc:1929] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.185874 29256 log.cc:1199] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/75642018d47b45e6ba74466c6d5ee95e
I20260809 07:58:46.186398 29256 ts_tablet_manager.cc:1950] T 75642018d47b45e6ba74466c6d5ee95e P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.186932 29085 ts_tablet_manager.cc:1929] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.187095 29085 log.cc:1199] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/33198e831822470db98234b54d09c6bc
I20260809 07:58:46.187646 29085 ts_tablet_manager.cc:1950] T 33198e831822470db98234b54d09c6bc P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.188859 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.189118 29257 ts_tablet_manager.cc:1929] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.189251 29257 log.cc:1199] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/3c44aace20a640ef8aca975a73af0d93
I20260809 07:58:46.189728 29257 ts_tablet_manager.cc:1950] T 3c44aace20a640ef8aca975a73af0d93 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.190505 29085 tablet_replica.cc:333] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.190723 29085 raft_consensus.cc:2243] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.190884 29257 tablet_replica.cc:333] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.191102 29256 tablet_replica.cc:333] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.191085 29257 raft_consensus.cc:2243] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.191104 29085 raft_consensus.cc:2272] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.191285 29256 raft_consensus.cc:2243] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.191424 29257 raft_consensus.cc:2272] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.191444 29256 raft_consensus.cc:2272] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.192209 29257 ts_tablet_manager.cc:1916] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.192749 29256 ts_tablet_manager.cc:1916] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.193611 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.194175 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 3c44aace20a640ef8aca975a73af0d93 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.198732 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 33198e831822470db98234b54d09c6bc (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.198905 29257 ts_tablet_manager.cc:1929] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.198979 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 75642018d47b45e6ba74466c6d5ee95e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.199007 29257 log.cc:1199] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/39664b79fe3246dfbae1cb687bfc8129
I20260809 07:58:46.199458 29257 ts_tablet_manager.cc:1950] T 39664b79fe3246dfbae1cb687bfc8129 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.200390 29256 ts_tablet_manager.cc:1929] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.200575 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 39664b79fe3246dfbae1cb687bfc8129 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.200645 29256 log.cc:1199] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/3c44aace20a640ef8aca975a73af0d93
I20260809 07:58:46.200955 29257 tablet_replica.cc:333] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.201172 29256 ts_tablet_manager.cc:1950] T 3c44aace20a640ef8aca975a73af0d93 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.201251 29257 raft_consensus.cc:2243] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.201570 29257 raft_consensus.cc:2272] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.203161 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 3c44aace20a640ef8aca975a73af0d93 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.203250 29257 ts_tablet_manager.cc:1916] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.203554 29256 tablet_replica.cc:333] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.203776 29256 raft_consensus.cc:2243] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.204044 29256 raft_consensus.cc:2272] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.204928 29256 ts_tablet_manager.cc:1916] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.207494 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.011s sys 0.001s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":43,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":996,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":964,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260809 07:58:46.208393 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:46.208712 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:46.208992 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:46.209307 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.268704
I20260809 07:58:46.209890 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260809 07:58:46.210294 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.210564 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.211704 29256 ts_tablet_manager.cc:1929] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.211818 29256 log.cc:1199] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/39664b79fe3246dfbae1cb687bfc8129
I20260809 07:58:46.212293 29256 ts_tablet_manager.cc:1950] T 39664b79fe3246dfbae1cb687bfc8129 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.213173 29085 ts_tablet_manager.cc:1916] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.213346 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 39664b79fe3246dfbae1cb687bfc8129 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.213562 29257 ts_tablet_manager.cc:1929] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.213703 29256 tablet_replica.cc:333] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.213729 29257 log.cc:1199] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/33198e831822470db98234b54d09c6bc
I20260809 07:58:46.213965 29256 raft_consensus.cc:2243] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.214191 29256 raft_consensus.cc:2272] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.214265 29257 ts_tablet_manager.cc:1950] T 33198e831822470db98234b54d09c6bc P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.215178 29256 ts_tablet_manager.cc:1916] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.215329 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 33198e831822470db98234b54d09c6bc (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.215684 29257 tablet_replica.cc:333] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.217015 29257 raft_consensus.cc:2243] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.217284 29257 raft_consensus.cc:2272] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.218184 29257 ts_tablet_manager.cc:1916] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.219559 29085 ts_tablet_manager.cc:1929] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.219691 29085 log.cc:1199] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/f47e81a4dd5943f7873144e373bab017
I20260809 07:58:46.224519 28941 consensus_queue.cc:1048] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.225903 29257 ts_tablet_manager.cc:1929] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.226082 29257 log.cc:1199] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/f47e81a4dd5943f7873144e373bab017
I20260809 07:58:46.226020 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.008s sys 0.005s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":45,"dirs.run_cpu_time_us":137,"dirs.run_wall_time_us":1443,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":981,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260809 07:58:46.226608 29257 ts_tablet_manager.cc:1950] T f47e81a4dd5943f7873144e373bab017 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.226979 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:46.227300 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.268859
I20260809 07:58:46.227924 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:46.228215 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:46.228775 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet f47e81a4dd5943f7873144e373bab017 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.228814 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20260809 07:58:46.229110 29257 tablet_replica.cc:333] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.229326 29257 raft_consensus.cc:2243] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.229521 29257 raft_consensus.cc:2272] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.231635 29256 ts_tablet_manager.cc:1929] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.231814 29256 log.cc:1199] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/f47e81a4dd5943f7873144e373bab017
I20260809 07:58:46.232161 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.232445 29256 ts_tablet_manager.cc:1950] T f47e81a4dd5943f7873144e373bab017 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.232504 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.233333 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet f47e81a4dd5943f7873144e373bab017 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.233670 29256 tablet_replica.cc:333] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.234011 29256 raft_consensus.cc:2243] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.234325 29256 raft_consensus.cc:2272] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.235191 29256 ts_tablet_manager.cc:1916] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.239326 29257 ts_tablet_manager.cc:1916] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.241084 29256 ts_tablet_manager.cc:1929] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.241215 29256 log.cc:1199] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/33198e831822470db98234b54d09c6bc
I20260809 07:58:46.241798 29256 ts_tablet_manager.cc:1950] T 33198e831822470db98234b54d09c6bc P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.242906 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 33198e831822470db98234b54d09c6bc (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.243252 29256 tablet_replica.cc:333] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.243485 29256 raft_consensus.cc:2243] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.243717 29256 raft_consensus.cc:2272] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.244634 29256 ts_tablet_manager.cc:1916] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.245014 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.245358 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.245733 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.245889 29257 ts_tablet_manager.cc:1929] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.247300 28602 tablet_service.cc:1511] Processing CreateTablet for tablet edd67bd7518a43ec94443bc1ed822a57 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.247614 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.247695 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd67bd7518a43ec94443bc1ed822a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.247879 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.247321 28422 tablet_service.cc:1511] Processing CreateTablet for tablet edd67bd7518a43ec94443bc1ed822a57 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.249220 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd67bd7518a43ec94443bc1ed822a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.251889 29256 ts_tablet_manager.cc:1929] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.252017 29256 log.cc:1199] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/193f851a4c1043e1af5aaf4b7190730d
I20260809 07:58:46.252483 29256 ts_tablet_manager.cc:1950] T 193f851a4c1043e1af5aaf4b7190730d P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.253573 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 193f851a4c1043e1af5aaf4b7190730d (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.253628 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.253885 29256 tablet_replica.cc:333] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.253875 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.253921 28788 tablet_service.cc:1511] Processing CreateTablet for tablet edd67bd7518a43ec94443bc1ed822a57 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.254069 29256 raft_consensus.cc:2243] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.254118 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.254336 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd67bd7518a43ec94443bc1ed822a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.254402 29256 raft_consensus.cc:2272] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.254871 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.003s sys 0.000s Metrics: {"bytes_written":705,"delete_count":0,"lbm_write_time_us":157,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":16}
I20260809 07:58:46.255160 29256 ts_tablet_manager.cc:1916] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.257573 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.257849 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.262243 29256 ts_tablet_manager.cc:1929] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.262374 29256 log.cc:1199] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/7751d1e95a124184848a095dfcc6268e
I20260809 07:58:46.262866 29256 ts_tablet_manager.cc:1950] T 7751d1e95a124184848a095dfcc6268e P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.263512 29001 consensus_queue.cc:1048] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.263969 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 7751d1e95a124184848a095dfcc6268e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.264288 29256 tablet_replica.cc:333] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.264492 29256 raft_consensus.cc:2243] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.264796 29256 raft_consensus.cc:2272] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.266656 28787 tablet_service.cc:1511] Processing CreateTablet for tablet e9e62dc7df994bc8ace03052264aa9a2 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.266844 28427 tablet_service.cc:1511] Processing CreateTablet for tablet e9e62dc7df994bc8ace03052264aa9a2 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.266891 28614 tablet_service.cc:1511] Processing CreateTablet for tablet e9e62dc7df994bc8ace03052264aa9a2 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.267114 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e62dc7df994bc8ace03052264aa9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.267230 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e62dc7df994bc8ace03052264aa9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.267318 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e62dc7df994bc8ace03052264aa9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.271219 28797 tablet_service.cc:1511] Processing CreateTablet for tablet c0862fc949b040b09e9dd945b1bc3e37 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.272065 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0862fc949b040b09e9dd945b1bc3e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.274673 28619 tablet_service.cc:1511] Processing CreateTablet for tablet c0862fc949b040b09e9dd945b1bc3e37 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.274794 28428 tablet_service.cc:1511] Processing CreateTablet for tablet c0862fc949b040b09e9dd945b1bc3e37 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.275058 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0862fc949b040b09e9dd945b1bc3e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.275166 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0862fc949b040b09e9dd945b1bc3e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.277102 29256 ts_tablet_manager.cc:1916] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.282709 29263 tablet_bootstrap.cc:492] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.284590 28619 tablet_service.cc:1511] Processing CreateTablet for tablet 0255a05235b3407d98a10bf04b0c8080 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.285115 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0255a05235b3407d98a10bf04b0c8080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.285008 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 0255a05235b3407d98a10bf04b0c8080 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.285456 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0255a05235b3407d98a10bf04b0c8080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.289978 29262 tablet_bootstrap.cc:492] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.292817 29262 tablet_bootstrap.cc:654] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.293776 29260 tablet_bootstrap.cc:492] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.246014 29257 log.cc:1199] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/193f851a4c1043e1af5aaf4b7190730d
I20260809 07:58:46.294337 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.004s sys 0.008s Metrics: {"bytes_written":705,"delete_count":0,"lbm_write_time_us":7638,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":3,"reinsert_count":0,"update_count":16}
I20260809 07:58:46.294564 29257 ts_tablet_manager.cc:1950] T 193f851a4c1043e1af5aaf4b7190730d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.295652 29257 tablet_replica.cc:333] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.295779 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 193f851a4c1043e1af5aaf4b7190730d (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.295866 29257 raft_consensus.cc:2243] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.296103 29257 raft_consensus.cc:2272] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.296068 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 4e7b2737e49c41438b6478656c6df0a6 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.289747 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 0255a05235b3407d98a10bf04b0c8080 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.296885 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e7b2737e49c41438b6478656c6df0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.297287 28797 tablet_service.cc:1511] Processing CreateTablet for tablet 4e7b2737e49c41438b6478656c6df0a6 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.296877 28602 tablet_service.cc:1511] Processing CreateTablet for tablet 4e7b2737e49c41438b6478656c6df0a6 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.298048 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 7dce89653b584c46a1128e0c9d43356d (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.298144 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e7b2737e49c41438b6478656c6df0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.298427 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dce89653b584c46a1128e0c9d43356d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.298735 28614 tablet_service.cc:1511] Processing CreateTablet for tablet 7dce89653b584c46a1128e0c9d43356d (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.299166 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dce89653b584c46a1128e0c9d43356d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.299691 29263 tablet_bootstrap.cc:654] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.304975 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0255a05235b3407d98a10bf04b0c8080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.305522 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e7b2737e49c41438b6478656c6df0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.305972 29257 ts_tablet_manager.cc:1916] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.309113 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 7dce89653b584c46a1128e0c9d43356d (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.309983 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dce89653b584c46a1128e0c9d43356d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.318588 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.319042 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.319281 29260 tablet_bootstrap.cc:654] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.320063 29263 tablet_bootstrap.cc:492] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.320282 29263 ts_tablet_manager.cc:1403] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.038s user 0.013s sys 0.000s
I20260809 07:58:46.319717 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.321024 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.321719 29256 ts_tablet_manager.cc:1929] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.321955 29256 log.cc:1199] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/12a98f94e5ed406ba4ac318dc3699913
I20260809 07:58:46.322296 29263 raft_consensus.cc:359] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.322444 29256 ts_tablet_manager.cc:1950] T 12a98f94e5ed406ba4ac318dc3699913 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.322515 29263 raft_consensus.cc:385] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.322640 29263 raft_consensus.cc:740] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.322938 29263 consensus_queue.cc:260] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.323475 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 12a98f94e5ed406ba4ac318dc3699913 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.323597 29263 ts_tablet_manager.cc:1434] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260809 07:58:46.324039 28602 tablet_service.cc:1511] Processing CreateTablet for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.325104 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.325434 29085 ts_tablet_manager.cc:1950] T f47e81a4dd5943f7873144e373bab017 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.325807 29257 ts_tablet_manager.cc:1929] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.325943 29257 log.cc:1199] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/12a98f94e5ed406ba4ac318dc3699913
I20260809 07:58:46.326448 29257 ts_tablet_manager.cc:1950] T 12a98f94e5ed406ba4ac318dc3699913 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.328827 29256 tablet_replica.cc:333] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.329041 29256 raft_consensus.cc:2243] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.329241 29256 raft_consensus.cc:2272] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.329237 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet f47e81a4dd5943f7873144e373bab017 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.330324 29256 ts_tablet_manager.cc:1916] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.330264 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 28a2e99f189940be8e9fb9981c740468 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.330642 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a2e99f189940be8e9fb9981c740468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.331144 29262 tablet_bootstrap.cc:492] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.331346 29262 ts_tablet_manager.cc:1403] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.041s user 0.005s sys 0.005s
I20260809 07:58:46.332266 29262 raft_consensus.cc:359] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.332543 29262 raft_consensus.cc:385] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.332700 29262 raft_consensus.cc:740] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.333034 29262 consensus_queue.cc:260] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.334945 29262 ts_tablet_manager.cc:1434] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:46.335279 29262 tablet_bootstrap.cc:492] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
W20260809 07:58:46.338533 28134 catalog_manager.cc:5017] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): delete failed for tablet 193f851a4c1043e1af5aaf4b7190730d because the tablet was not found. No further retry: Not found: Tablet not found: 193f851a4c1043e1af5aaf4b7190730d
I20260809 07:58:46.338912 29085 tablet_replica.cc:333] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.339112 29085 raft_consensus.cc:2243] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.339314 29085 raft_consensus.cc:2272] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.339774 29001 raft_consensus.cc:493] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.339936 29001 raft_consensus.cc:515] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.340327 29085 ts_tablet_manager.cc:1916] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.340617 29001 leader_election.cc:290] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223)
I20260809 07:58:46.340874 29263 tablet_bootstrap.cc:492] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.341392 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd67bd7518a43ec94443bc1ed822a57" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.341629 28648 raft_consensus.cc:2468] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:46.341977 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd67bd7518a43ec94443bc1ed822a57" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.342674 29263 tablet_bootstrap.cc:654] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
W20260809 07:58:46.343129 28740 leader_election.cc:343] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.343406 28740 leader_election.cc:304] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:46.343696 29001 raft_consensus.cc:2804] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.343823 29001 raft_consensus.cc:493] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.343936 29001 raft_consensus.cc:3060] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.341512 29262 tablet_bootstrap.cc:654] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.346669 29001 raft_consensus.cc:515] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.347342 29001 leader_election.cc:290] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223)
I20260809 07:58:46.347996 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd67bd7518a43ec94443bc1ed822a57" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:46.348157 29260 tablet_bootstrap.cc:492] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.348352 29260 ts_tablet_manager.cc:1403] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.055s user 0.000s sys 0.004s
I20260809 07:58:46.348471 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd67bd7518a43ec94443bc1ed822a57" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:46.348680 28648 raft_consensus.cc:3060] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.349278 29260 raft_consensus.cc:359] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.349756 29260 raft_consensus.cc:385] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.349917 29260 raft_consensus.cc:740] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.350227 29260 consensus_queue.cc:260] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.351622 28648 raft_consensus.cc:2468] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:46.351954 29262 tablet_bootstrap.cc:492] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.351940 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 7592e4276d0d4ddbb0d59cfec840984d (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.352129 29262 ts_tablet_manager.cc:1403] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.002s
I20260809 07:58:46.352164 29260 ts_tablet_manager.cc:1434] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:46.352376 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7592e4276d0d4ddbb0d59cfec840984d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.352445 29260 tablet_bootstrap.cc:492] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.352902 28740 leader_election.cc:304] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:46.352933 29262 raft_consensus.cc:359] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.353122 29262 raft_consensus.cc:385] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.353206 29262 raft_consensus.cc:740] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
W20260809 07:58:46.353257 28740 leader_election.cc:343] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.353410 29001 raft_consensus.cc:2804] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.353765 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 28a2e99f189940be8e9fb9981c740468 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.353897 29001 raft_consensus.cc:697] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:46.354198 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a2e99f189940be8e9fb9981c740468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.354198 29001 consensus_queue.cc:237] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.356810 29260 tablet_bootstrap.cc:654] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.357528 29262 consensus_queue.cc:260] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.360615 29262 ts_tablet_manager.cc:1434] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260809 07:58:46.361307 29262 tablet_bootstrap.cc:492] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.364848 29257 tablet_replica.cc:333] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.365078 29257 raft_consensus.cc:2243] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.365398 29257 raft_consensus.cc:2272] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.365371 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 12a98f94e5ed406ba4ac318dc3699913 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.365765 28422 tablet_service.cc:1511] Processing CreateTablet for tablet dafaca6dde5b412e83e9f4cc1edb1f66 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.366420 29257 ts_tablet_manager.cc:1916] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.366658 28797 tablet_service.cc:1511] Processing CreateTablet for tablet dafaca6dde5b412e83e9f4cc1edb1f66 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.367063 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafaca6dde5b412e83e9f4cc1edb1f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.367138 28614 tablet_service.cc:1511] Processing CreateTablet for tablet dafaca6dde5b412e83e9f4cc1edb1f66 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.367239 28602 tablet_service.cc:1511] Processing CreateTablet for tablet 7592e4276d0d4ddbb0d59cfec840984d (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.367540 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafaca6dde5b412e83e9f4cc1edb1f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.367831 28603 tablet_service.cc:1511] Processing CreateTablet for tablet f152550cc1194a04962055b95dee8e78 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.367914 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafaca6dde5b412e83e9f4cc1edb1f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.368193 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f152550cc1194a04962055b95dee8e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.370234 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2502 bytes on disk
I20260809 07:58:46.370491 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2502 bytes on disk
I20260809 07:58:46.370718 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2502 bytes on disk
I20260809 07:58:46.371306 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260809 07:58:46.371709 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.372010 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.372437 28429 tablet_service.cc:1511] Processing CreateTablet for tablet f152550cc1194a04962055b95dee8e78 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.372884 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f152550cc1194a04962055b95dee8e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.373610 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7592e4276d0d4ddbb0d59cfec840984d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.374380 29262 tablet_bootstrap.cc:654] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.375269 29256 ts_tablet_manager.cc:1929] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.376426 29256 log.cc:1199] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/2a54a36e22474698a198d8c25bafa16b
I20260809 07:58:46.376780 28619 tablet_service.cc:1511] Processing CreateTablet for tablet 28a2e99f189940be8e9fb9981c740468 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.376899 29256 ts_tablet_manager.cc:1950] T 2a54a36e22474698a198d8c25bafa16b P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.377302 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a2e99f189940be8e9fb9981c740468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.378206 29256 tablet_replica.cc:333] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.378546 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 2a54a36e22474698a198d8c25bafa16b (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.378623 29256 raft_consensus.cc:2243] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.378800 29256 raft_consensus.cc:2272] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 07:58:46.378855 28134 catalog_manager.cc:5017] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): delete failed for tablet 7751d1e95a124184848a095dfcc6268e because the tablet was not found. No further retry: Not found: Tablet not found: 7751d1e95a124184848a095dfcc6268e
I20260809 07:58:46.379850 29256 ts_tablet_manager.cc:1916] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.384276 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 7592e4276d0d4ddbb0d59cfec840984d (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.384821 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7592e4276d0d4ddbb0d59cfec840984d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.386698 29257 ts_tablet_manager.cc:1929] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.386868 29257 log.cc:1199] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/2a54a36e22474698a198d8c25bafa16b
I20260809 07:58:46.387364 29257 ts_tablet_manager.cc:1950] T 2a54a36e22474698a198d8c25bafa16b P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.387658 29263 tablet_bootstrap.cc:492] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.387883 29263 ts_tablet_manager.cc:1403] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20260809 07:58:46.388481 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 2a54a36e22474698a198d8c25bafa16b (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.389014 29263 raft_consensus.cc:359] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.391538 28787 tablet_service.cc:1511] Processing CreateTablet for tablet f152550cc1194a04962055b95dee8e78 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.391952 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f152550cc1194a04962055b95dee8e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.389283 29263 raft_consensus.cc:385] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.392797 29263 raft_consensus.cc:740] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.393155 29263 consensus_queue.cc:260] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.393240 28797 tablet_service.cc:1511] Processing CreateTablet for tablet ba22ea1e78504b34ba032ce8575f0890 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.393685 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba22ea1e78504b34ba032ce8575f0890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.394315 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 744b57379dd54528a734afcedf8128b7 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.394775 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744b57379dd54528a734afcedf8128b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.395085 28795 tablet_service.cc:1511] Processing CreateTablet for tablet 744b57379dd54528a734afcedf8128b7 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.395535 28795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744b57379dd54528a734afcedf8128b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.396464 29260 tablet_bootstrap.cc:492] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.396641 29260 ts_tablet_manager.cc:1403] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.002s
I20260809 07:58:46.402432 29263 ts_tablet_manager.cc:1434] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260809 07:58:46.402590 28428 tablet_service.cc:1511] Processing CreateTablet for tablet ba22ea1e78504b34ba032ce8575f0890 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.402837 29263 tablet_bootstrap.cc:492] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.403050 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba22ea1e78504b34ba032ce8575f0890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.403301 29260 raft_consensus.cc:359] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.403564 29260 raft_consensus.cc:385] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.403640 29260 raft_consensus.cc:740] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.403967 29260 consensus_queue.cc:260] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.405615 29260 ts_tablet_manager.cc:1434] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260809 07:58:46.405921 29260 tablet_bootstrap.cc:492] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.406394 28149 catalog_manager.cc:5719] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:46.407963 29257 tablet_replica.cc:333] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.407982 29263 tablet_bootstrap.cc:654] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.408205 29257 raft_consensus.cc:2243] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.408424 29257 raft_consensus.cc:2272] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.409085 29260 tablet_bootstrap.cc:654] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.415158 29257 ts_tablet_manager.cc:1916] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.423514 29085 ts_tablet_manager.cc:1929] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.423836 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.424166 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.424455 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.424732 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 0bad7eb37a784618b13b2263c0215b21 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.425240 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bad7eb37a784618b13b2263c0215b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.426301 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.426570 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.427079 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.427330 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.427470 29263 tablet_bootstrap.cc:492] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.427603 29263 ts_tablet_manager.cc:1403] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.005s
I20260809 07:58:46.427656 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.428395 29263 raft_consensus.cc:359] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.428591 29263 raft_consensus.cc:385] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.428696 29263 raft_consensus.cc:740] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.428944 29263 consensus_queue.cc:260] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.429095 28602 tablet_service.cc:1511] Processing CreateTablet for tablet ba22ea1e78504b34ba032ce8575f0890 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.429512 29263 ts_tablet_manager.cc:1434] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:46.430303 28614 tablet_service.cc:1511] Processing CreateTablet for tablet 744b57379dd54528a734afcedf8128b7 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.430712 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744b57379dd54528a734afcedf8128b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.431056 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 1b65040b5e344742890ad101393efc48 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.431388 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b65040b5e344742890ad101393efc48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.431629 28603 tablet_service.cc:1511] Processing CreateTablet for tablet 0bad7eb37a784618b13b2263c0215b21 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.431983 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bad7eb37a784618b13b2263c0215b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.434530 29256 ts_tablet_manager.cc:1929] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.434671 29256 log.cc:1199] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/fc1ec621e45e4e57b7fb536f3db739eb
I20260809 07:58:46.435225 29256 ts_tablet_manager.cc:1950] T fc1ec621e45e4e57b7fb536f3db739eb P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.437886 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba22ea1e78504b34ba032ce8575f0890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.438639 29263 tablet_bootstrap.cc:492] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.439262 28620 tablet_service.cc:1511] Processing CreateTablet for tablet 1b65040b5e344742890ad101393efc48 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.439718 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b65040b5e344742890ad101393efc48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.440497 29263 tablet_bootstrap.cc:654] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.441684 28797 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad6dd9c01dd45f3a16a9be616205018 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.442101 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad6dd9c01dd45f3a16a9be616205018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.442327 29085 log.cc:1199] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/12a98f94e5ed406ba4ac318dc3699913
I20260809 07:58:46.442826 29085 ts_tablet_manager.cc:1950] T 12a98f94e5ed406ba4ac318dc3699913 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.443838 29085 tablet_replica.cc:333] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.444015 29085 raft_consensus.cc:2243] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.444196 29085 raft_consensus.cc:2272] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.439841 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 0bad7eb37a784618b13b2263c0215b21 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.444634 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bad7eb37a784618b13b2263c0215b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.441990 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 1b65040b5e344742890ad101393efc48 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.446655 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b65040b5e344742890ad101393efc48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.448706 29256 tablet_replica.cc:333] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.448979 29256 raft_consensus.cc:2243] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.449225 29256 raft_consensus.cc:2272] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.449667 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.002s sys 0.003s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":167,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.451639 29256 ts_tablet_manager.cc:1916] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.451778 29263 tablet_bootstrap.cc:492] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.451975 29263 ts_tablet_manager.cc:1403] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.001s
I20260809 07:58:46.451817 29262 tablet_bootstrap.cc:492] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.452572 29262 ts_tablet_manager.cc:1403] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.092s user 0.000s sys 0.013s
I20260809 07:58:46.452615 29263 raft_consensus.cc:359] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.452822 29263 raft_consensus.cc:385] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.452912 29263 raft_consensus.cc:740] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.453204 29263 consensus_queue.cc:260] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.453536 29262 raft_consensus.cc:359] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.453722 29262 raft_consensus.cc:385] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.453819 29262 raft_consensus.cc:740] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.454469 29263 ts_tablet_manager.cc:1434] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:46.454790 29263 tablet_bootstrap.cc:492] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.455408 29262 consensus_queue.cc:260] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.457041 29263 tablet_bootstrap.cc:654] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.458153 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.247s user 0.029s sys 0.005s Metrics: {"bytes_written":705,"cfile_cache_hit":16,"cfile_cache_hit_bytes":15246,"cfile_cache_miss":42,"cfile_cache_miss_bytes":24120,"cfile_init":12,"delete_count":0,"delta_iterators_relevant":13,"dirs.queue_time_us":58318,"dirs.run_cpu_time_us":1389,"dirs.run_wall_time_us":9669,"drs_written":1,"lbm_read_time_us":1771,"lbm_reads_lt_1ms":90,"lbm_write_time_us":1174,"lbm_writes_lt_1ms":31,"mutex_wait_us":29060,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":54,"thread_start_us":36283,"threads_started":2,"update_count":16}
I20260809 07:58:46.460729 29262 ts_tablet_manager.cc:1434] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260809 07:58:46.462726 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.006s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":939,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":37,"lbm_read_time_us":181,"lbm_reads_lt_1ms":8,"lbm_write_time_us":163,"lbm_writes_lt_1ms":4,"mutex_wait_us":18528,"reinsert_count":0,"update_count":17}
I20260809 07:58:46.463387 29262 tablet_bootstrap.cc:492] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.463919 29256 ts_tablet_manager.cc:1929] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.464119 29256 log.cc:1199] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/ab1a97f98f5446daa2523cf10fb8fac4
I20260809 07:58:46.464280 29001 raft_consensus.cc:493] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.464459 29001 raft_consensus.cc:515] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.464622 29260 tablet_bootstrap.cc:492] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.464769 29260 ts_tablet_manager.cc:1403] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.059s user 0.003s sys 0.001s
I20260809 07:58:46.464939 29256 ts_tablet_manager.cc:1950] T ab1a97f98f5446daa2523cf10fb8fac4 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.465181 29001 leader_election.cc:290] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:46.465626 29260 raft_consensus.cc:359] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.465833 29260 raft_consensus.cc:385] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.465942 29260 raft_consensus.cc:740] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.466214 29260 consensus_queue.cc:260] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.466432 29256 tablet_replica.cc:333] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.466380 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0862fc949b040b09e9dd945b1bc3e37" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.466603 28464 raft_consensus.cc:2468] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:46.466784 29256 raft_consensus.cc:2243] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.467036 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0862fc949b040b09e9dd945b1bc3e37" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.467247 28648 raft_consensus.cc:2468] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:46.467262 29256 raft_consensus.cc:2272] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.467690 28740 leader_election.cc:304] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:46.468078 29260 ts_tablet_manager.cc:1434] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:46.468389 29260 tablet_bootstrap.cc:492] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.468693 29256 ts_tablet_manager.cc:1916] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.468885 29262 tablet_bootstrap.cc:654] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.470273 29260 tablet_bootstrap.cc:654] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.470605 29001 raft_consensus.cc:2804] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.470714 29001 raft_consensus.cc:493] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.470835 29001 raft_consensus.cc:3060] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.472739 29257 ts_tablet_manager.cc:1929] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.473040 29257 log.cc:1199] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/7751d1e95a124184848a095dfcc6268e
I20260809 07:58:46.473532 29257 ts_tablet_manager.cc:1950] T 7751d1e95a124184848a095dfcc6268e P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.474478 29257 tablet_replica.cc:333] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.474728 29257 raft_consensus.cc:2243] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.475087 29257 raft_consensus.cc:2272] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.477504 29262 tablet_bootstrap.cc:492] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.477675 29262 ts_tablet_manager.cc:1403] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.002s
I20260809 07:58:46.477682 29256 ts_tablet_manager.cc:1929] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.477972 29256 log.cc:1199] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/550838d288be4ecf8a9dd10aeda6467a
I20260809 07:58:46.478447 29256 ts_tablet_manager.cc:1950] T 550838d288be4ecf8a9dd10aeda6467a P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.478375 29262 raft_consensus.cc:359] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.478624 29262 raft_consensus.cc:385] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.478716 29262 raft_consensus.cc:740] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.478965 29262 consensus_queue.cc:260] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.474264 29001 raft_consensus.cc:515] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.479838 29001 leader_election.cc:290] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:46.480285 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0862fc949b040b09e9dd945b1bc3e37" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:46.480484 28464 raft_consensus.cc:3060] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.481177 29256 tablet_replica.cc:333] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.481693 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0862fc949b040b09e9dd945b1bc3e37" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:46.481777 29256 raft_consensus.cc:2243] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.481866 28648 raft_consensus.cc:3060] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.482143 29256 raft_consensus.cc:2272] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.482204 29257 ts_tablet_manager.cc:1916] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.483093 29085 ts_tablet_manager.cc:1916] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.484894 28648 raft_consensus.cc:2468] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:46.485623 28740 leader_election.cc:304] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:46.485924 29001 raft_consensus.cc:2804] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.486081 29001 raft_consensus.cc:697] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:46.486346 29001 consensus_queue.cc:237] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.487782 29260 tablet_bootstrap.cc:492] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.488008 29260 ts_tablet_manager.cc:1403] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260809 07:58:46.488837 29260 raft_consensus.cc:359] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.488868 28464 raft_consensus.cc:2468] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:46.489387 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad6dd9c01dd45f3a16a9be616205018 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.489881 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad6dd9c01dd45f3a16a9be616205018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.489801 28603 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad6dd9c01dd45f3a16a9be616205018 (DEFAULT_TABLE table=table-2 [id=fc96e4603f5148449b4d5a5358162240]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:46.490151 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad6dd9c01dd45f3a16a9be616205018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:46.493347 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet fc1ec621e45e4e57b7fb536f3db739eb (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.494248 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet ab1a97f98f5446daa2523cf10fb8fac4 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.494530 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 550838d288be4ecf8a9dd10aeda6467a (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.494946 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 12a98f94e5ed406ba4ac318dc3699913 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
W20260809 07:58:46.495337 28135 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.26.37.3:38449, user_credentials={real_user=slave}} blocked reactor thread for 50435.5us
I20260809 07:58:46.495978 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 7751d1e95a124184848a095dfcc6268e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.497360 29260 raft_consensus.cc:385] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.497526 29260 raft_consensus.cc:740] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.497872 29260 consensus_queue.cc:260] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.499755 29263 tablet_bootstrap.cc:492] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.499781 29256 ts_tablet_manager.cc:1916] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.499898 29263 ts_tablet_manager.cc:1403] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.045s user 0.009s sys 0.001s
I20260809 07:58:46.499020 29260 ts_tablet_manager.cc:1434] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260809 07:58:46.500777 29263 raft_consensus.cc:359] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.500963 29263 raft_consensus.cc:385] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.501048 29263 raft_consensus.cc:740] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.501297 29263 consensus_queue.cc:260] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.501578 29262 ts_tablet_manager.cc:1434] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
I20260809 07:58:46.501916 29263 ts_tablet_manager.cc:1434] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:46.502146 29260 tablet_bootstrap.cc:492] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.502216 29263 tablet_bootstrap.cc:492] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.503389 29262 tablet_bootstrap.cc:492] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.504102 29263 tablet_bootstrap.cc:654] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.504384 28941 raft_consensus.cc:493] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.504549 28941 raft_consensus.cc:515] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.505162 28941 leader_election.cc:290] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:46.506228 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e7b2737e49c41438b6478656c6df0a6" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.506299 29260 tablet_bootstrap.cc:654] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
W20260809 07:58:46.506775 28740 leader_election.cc:343] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.507089 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e7b2737e49c41438b6478656c6df0a6" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
W20260809 07:58:46.508831 28740 leader_election.cc:343] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.508965 28740 leader_election.cc:304] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:46.509445 28941 raft_consensus.cc:2749] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:46.511674 28961 raft_consensus.cc:493] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.511852 28961 raft_consensus.cc:515] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.512600 28961 leader_election.cc:290] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.513460 29085 ts_tablet_manager.cc:1929] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.513772 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e62dc7df994bc8ace03052264aa9a2" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:46.513965 28826 raft_consensus.cc:2468] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:46.514570 28566 leader_election.cc:304] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:46.515101 29262 tablet_bootstrap.cc:654] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.515478 28961 raft_consensus.cc:2804] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.515628 28961 raft_consensus.cc:493] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.515743 28961 raft_consensus.cc:3060] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.517885 29256 ts_tablet_manager.cc:1929] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.518028 29256 log.cc:1199] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/a48541470e51457d980a8c3ea27b060f
I20260809 07:58:46.518546 29256 ts_tablet_manager.cc:1950] T a48541470e51457d980a8c3ea27b060f P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.518657 28961 raft_consensus.cc:515] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.519145 28149 catalog_manager.cc:5719] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:46.520328 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2502 bytes on disk
I20260809 07:58:46.520714 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2502 bytes on disk
I20260809 07:58:46.520987 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2502 bytes on disk
I20260809 07:58:46.521377 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"spinlock_wait_cycles":7552}
I20260809 07:58:46.521596 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.521854 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.522140 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e62dc7df994bc8ace03052264aa9a2" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:46.522336 28826 raft_consensus.cc:3060] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.524931 28826 raft_consensus.cc:2468] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:46.526094 28566 leader_election.cc:304] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:46.526409 29029 raft_consensus.cc:2804] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.526544 29029 raft_consensus.cc:697] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:46.526859 29029 consensus_queue.cc:237] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.527769 29257 ts_tablet_manager.cc:1929] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.527925 29257 log.cc:1199] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/fc1ec621e45e4e57b7fb536f3db739eb
I20260809 07:58:46.528791 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.529167 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.529517 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet a48541470e51457d980a8c3ea27b060f (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.529632 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.530634 29256 tablet_replica.cc:333] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:46.530826 29256 raft_consensus.cc:2243] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.530965 29256 raft_consensus.cc:2272] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.531436 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101651
I20260809 07:58:46.531792 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101651
I20260809 07:58:46.532220 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.532583 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.532868 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.533951 28961 leader_election.cc:290] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.538228 29256 ts_tablet_manager.cc:1916] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.541683 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.541827 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.000s sys 0.005s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":182,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.542002 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.542280 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.542553 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.544224 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.544467 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.544704 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.544772 29262 tablet_bootstrap.cc:492] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.544914 29262 ts_tablet_manager.cc:1403] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.004s
I20260809 07:58:46.545774 29262 raft_consensus.cc:359] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.545979 29262 raft_consensus.cc:385] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.546079 29262 raft_consensus.cc:740] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.546319 29262 consensus_queue.cc:260] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.546787 29262 ts_tablet_manager.cc:1434] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:46.547062 29262 tablet_bootstrap.cc:492] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.548094 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.012s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":14210,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":95,"lbm_read_time_us":494,"lbm_reads_lt_1ms":23,"lbm_write_time_us":370,"lbm_writes_lt_1ms":9,"mutex_wait_us":2840,"peak_mem_usage":32768,"reinsert_count":0,"update_count":18}
I20260809 07:58:46.548776 29262 tablet_bootstrap.cc:654] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.549790 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101651
I20260809 07:58:46.550058 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101651
I20260809 07:58:46.550406 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101651
I20260809 07:58:46.550769 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.551029 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.551357 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.551774 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.553516 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.004s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":151,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.560235 29257 ts_tablet_manager.cc:1950] T fc1ec621e45e4e57b7fb536f3db739eb P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.553232 29256 ts_tablet_manager.cc:1929] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.562067 29256 log.cc:1199] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/eff7ca7c344b48b685c829148b184b9b
I20260809 07:58:46.562093 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet fc1ec621e45e4e57b7fb536f3db739eb (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.562590 29256 ts_tablet_manager.cc:1950] T eff7ca7c344b48b685c829148b184b9b P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:46.563639 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet eff7ca7c344b48b685c829148b184b9b (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.561903 29257 tablet_replica.cc:333] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.564893 29257 raft_consensus.cc:2243] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.565784 29257 raft_consensus.cc:2272] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.566515 29262 tablet_bootstrap.cc:492] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.566679 29262 ts_tablet_manager.cc:1403] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.003s
I20260809 07:58:46.567543 29262 raft_consensus.cc:359] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.568532 29257 ts_tablet_manager.cc:1916] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.568241 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.011s sys 0.003s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":14210,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":977,"dirs.run_cpu_time_us":288,"dirs.run_wall_time_us":1397,"lbm_read_time_us":430,"lbm_reads_lt_1ms":23,"lbm_write_time_us":391,"lbm_writes_lt_1ms":9,"mutex_wait_us":1645,"peak_mem_usage":32768,"reinsert_count":0,"update_count":18}
I20260809 07:58:46.513913 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e62dc7df994bc8ace03052264aa9a2" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.568921 28464 raft_consensus.cc:2468] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:46.519624 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e62dc7df994bc8ace03052264aa9a2" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:46.569293 28461 raft_consensus.cc:3060] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.570518 29085 log.cc:1199] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/2a54a36e22474698a198d8c25bafa16b
I20260809 07:58:46.571102 29085 ts_tablet_manager.cc:1950] T 2a54a36e22474698a198d8c25bafa16b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.572398 28461 raft_consensus.cc:2468] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:46.572579 29263 tablet_bootstrap.cc:492] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.572770 29263 ts_tablet_manager.cc:1403] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.071s user 0.004s sys 0.000s
I20260809 07:58:46.521474 29014 raft_consensus.cc:493] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.572906 29085 tablet_replica.cc:333] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.572995 29014 raft_consensus.cc:515] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.573172 29085 raft_consensus.cc:2243] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.573426 29085 raft_consensus.cc:2272] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.573544 29263 raft_consensus.cc:359] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.573729 29014 leader_election.cc:290] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.573752 29263 raft_consensus.cc:385] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.573845 29263 raft_consensus.cc:740] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.574108 29263 consensus_queue.cc:260] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.574327 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd67bd7518a43ec94443bc1ed822a57" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:46.574702 29263 ts_tablet_manager.cc:1434] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:46.575065 29263 tablet_bootstrap.cc:492] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.574967 28149 catalog_manager.cc:5719] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:46.576141 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 2a54a36e22474698a198d8c25bafa16b (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.576794 29263 tablet_bootstrap.cc:654] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.579829 29262 raft_consensus.cc:385] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.579983 29262 raft_consensus.cc:740] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.580230 29262 consensus_queue.cc:260] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.580374 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd67bd7518a43ec94443bc1ed822a57" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.580592 28648 raft_consensus.cc:2393] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in current term 1: Already voted for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in this term.
I20260809 07:58:46.580827 29262 ts_tablet_manager.cc:1434] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260809 07:58:46.581183 29262 tablet_bootstrap.cc:492] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.581416 28380 leader_election.cc:304] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:46.582000 29014 raft_consensus.cc:3060] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.584793 29014 raft_consensus.cc:2749] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:46.585021 28941 raft_consensus.cc:493] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.585210 28941 raft_consensus.cc:515] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.585968 28941 leader_election.cc:290] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:46.587118 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dce89653b584c46a1128e0c9d43356d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.587350 28648 raft_consensus.cc:2468] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:46.587538 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dce89653b584c46a1128e0c9d43356d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.587898 28740 leader_election.cc:304] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
W20260809 07:58:46.588387 28740 leader_election.cc:343] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.588754 28941 raft_consensus.cc:2804] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.588863 28941 raft_consensus.cc:493] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.588964 28941 raft_consensus.cc:3060] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.591182 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.591521 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.591758 28941 raft_consensus.cc:515] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.591853 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.592597 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dce89653b584c46a1128e0c9d43356d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:46.559687 29260 tablet_bootstrap.cc:492] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.593040 29260 ts_tablet_manager.cc:1403] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.091s user 0.001s sys 0.003s
I20260809 07:58:46.593322 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dce89653b584c46a1128e0c9d43356d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:46.593532 28648 raft_consensus.cc:3060] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.593922 29260 raft_consensus.cc:359] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.594030 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.594125 29260 raft_consensus.cc:385] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.594208 29260 raft_consensus.cc:740] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.594368 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.594415 29260 consensus_queue.cc:260] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.594883 29260 ts_tablet_manager.cc:1434] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:46.596380 28648 raft_consensus.cc:2468] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:46.596480 29260 tablet_bootstrap.cc:492] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.597016 28740 leader_election.cc:304] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:46.597520 29211 raft_consensus.cc:2804] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.597677 29211 raft_consensus.cc:697] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:46.597918 29211 consensus_queue.cc:237] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.598388 29260 tablet_bootstrap.cc:654] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.599696 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
W20260809 07:58:46.599701 28740 leader_election.cc:343] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.600020 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.600313 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.600497 28941 leader_election.cc:290] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:46.600659 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.600941 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.587348 29262 tablet_bootstrap.cc:654] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.602774 28961 raft_consensus.cc:493] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.607828 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.002s sys 0.003s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":160,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.611506 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.611130 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.382s user 0.039s sys 0.028s Metrics: {"bytes_written":795,"cfile_cache_hit":16,"cfile_cache_hit_bytes":15246,"cfile_cache_miss":42,"cfile_cache_miss_bytes":24052,"cfile_init":11,"delete_count":0,"delta_iterators_relevant":13,"dirs.queue_time_us":1892,"dirs.run_cpu_time_us":1325,"dirs.run_wall_time_us":7313,"drs_written":1,"lbm_read_time_us":3906,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":85,"lbm_write_time_us":38677,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":30,"mutex_wait_us":1159,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":54,"thread_start_us":48752,"threads_started":2,"update_count":17}
I20260809 07:58:46.612123 29085 ts_tablet_manager.cc:1916] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.618530 28961 raft_consensus.cc:515] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.619773 29085 ts_tablet_manager.cc:1929] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.619933 29085 log.cc:1199] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/7751d1e95a124184848a095dfcc6268e
I20260809 07:58:46.620447 29085 ts_tablet_manager.cc:1950] T 7751d1e95a124184848a095dfcc6268e P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.620954 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255a05235b3407d98a10bf04b0c8080" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.621172 28461 raft_consensus.cc:2468] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:46.621241 28961 leader_election.cc:290] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.621325 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255a05235b3407d98a10bf04b0c8080" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:46.621592 28826 raft_consensus.cc:2468] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:46.621860 28565 leader_election.cc:304] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.622255 28961 raft_consensus.cc:2804] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.622608 29085 tablet_replica.cc:333] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.622701 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 7751d1e95a124184848a095dfcc6268e (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.622812 29085 raft_consensus.cc:2243] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.623011 29085 raft_consensus.cc:2272] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.623819 29085 ts_tablet_manager.cc:1916] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.624320 28961 raft_consensus.cc:493] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.624461 28961 raft_consensus.cc:3060] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.627602 28961 raft_consensus.cc:515] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.628677 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255a05235b3407d98a10bf04b0c8080" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:46.628873 28461 raft_consensus.cc:3060] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.629231 28149 catalog_manager.cc:5719] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:46.630514 29085 ts_tablet_manager.cc:1929] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.630633 29085 log.cc:1199] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/fc1ec621e45e4e57b7fb536f3db739eb
I20260809 07:58:46.631088 29085 ts_tablet_manager.cc:1950] T fc1ec621e45e4e57b7fb536f3db739eb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.631846 28461 raft_consensus.cc:2468] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:46.632023 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet fc1ec621e45e4e57b7fb536f3db739eb (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.632105 29085 tablet_replica.cc:333] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.632308 29085 raft_consensus.cc:2243] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.632477 28565 leader_election.cc:304] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.633097 28961 leader_election.cc:290] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.633400 28961 raft_consensus.cc:2804] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.633611 28961 raft_consensus.cc:697] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:46.633679 29263 tablet_bootstrap.cc:492] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.633878 29263 ts_tablet_manager.cc:1403] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.059s user 0.006s sys 0.003s
I20260809 07:58:46.634045 28961 consensus_queue.cc:237] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.634478 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255a05235b3407d98a10bf04b0c8080" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:46.634660 28826 raft_consensus.cc:3060] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.637744 29263 raft_consensus.cc:359] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.637956 29263 raft_consensus.cc:385] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.638096 29263 raft_consensus.cc:740] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.638661 29263 consensus_queue.cc:260] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.641898 29260 tablet_bootstrap.cc:492] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.642087 29260 ts_tablet_manager.cc:1403] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.046s user 0.015s sys 0.013s
I20260809 07:58:46.642655 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.642859 29260 raft_consensus.cc:359] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.643025 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.643080 29260 raft_consensus.cc:385] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.643220 29260 raft_consensus.cc:740] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.643509 29260 consensus_queue.cc:260] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.643908 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.644253 28149 catalog_manager.cc:5719] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:46.644557 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.644866 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.645140 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.645458 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.645736 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.647899 29263 ts_tablet_manager.cc:1434] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.014s user 0.000s sys 0.002s
I20260809 07:58:46.648214 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.648550 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.648828 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.649055 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.650719 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.651697 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.656438 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.656656 29263 tablet_bootstrap.cc:492] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.656772 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.657297 29211 raft_consensus.cc:493] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.657464 29211 raft_consensus.cc:515] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.658155 29211 leader_election.cc:290] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:46.658979 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a2e99f189940be8e9fb9981c740468" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.659073 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a2e99f189940be8e9fb9981c740468" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
W20260809 07:58:46.659664 28740 leader_election.cc:343] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:46.659912 28740 leader_election.cc:343] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.660076 28740 leader_election.cc:304] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:46.660534 29211 raft_consensus.cc:2749] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:46.660701 28826 raft_consensus.cc:2468] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:46.661150 29263 tablet_bootstrap.cc:654] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.661298 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.661542 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.005s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":157,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.661940 29257 ts_tablet_manager.cc:1929] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.662087 29257 log.cc:1199] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/ab1a97f98f5446daa2523cf10fb8fac4
I20260809 07:58:46.662060 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.662191 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.003000
I20260809 07:58:46.662370 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":11287}
I20260809 07:58:46.662331 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":187,"lbm_writes_lt_1ms":4,"mutex_wait_us":4253,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.662602 29257 ts_tablet_manager.cc:1950] T ab1a97f98f5446daa2523cf10fb8fac4 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.663630 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":141,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.664167 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet ab1a97f98f5446daa2523cf10fb8fac4 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.664280 29260 ts_tablet_manager.cc:1434] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.022s user 0.000s sys 0.002s
W20260809 07:58:46.664683 28740 consensus_peers.cc:597] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a48541470e51457d980a8c3ea27b060f. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:46.665961 29260 tablet_bootstrap.cc:492] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.667599 29257 tablet_replica.cc:333] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.667837 29257 raft_consensus.cc:2243] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.667804 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":10589}
I20260809 07:58:46.668051 29260 tablet_bootstrap.cc:654] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.668051 29257 raft_consensus.cc:2272] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.669487 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.001s sys 0.004s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":138,"lbm_writes_lt_1ms":4,"mutex_wait_us":5996,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.669947 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":7067}
I20260809 07:58:46.670333 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":670,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":2509,"dirs.run_cpu_time_us":167,"dirs.run_wall_time_us":3466,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4,"lbm_write_time_us":131,"lbm_writes_lt_1ms":4,"mutex_wait_us":8568,"reinsert_count":0,"update_count":2}
I20260809 07:58:46.670897 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.671208 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.671129 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":6,"cfile_cache_miss_bytes":1005,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":50,"lbm_read_time_us":286,"lbm_reads_lt_1ms":14,"lbm_write_time_us":154,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260809 07:58:46.671487 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.671828 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.672492 29257 ts_tablet_manager.cc:1916] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.674162 29262 tablet_bootstrap.cc:492] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.674314 29262 ts_tablet_manager.cc:1403] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.093s user 0.003s sys 0.000s
I20260809 07:58:46.675146 29262 raft_consensus.cc:359] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.675350 29262 raft_consensus.cc:385] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.675437 29262 raft_consensus.cc:740] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.675701 29262 consensus_queue.cc:260] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.676211 29262 ts_tablet_manager.cc:1434] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 07:58:46.676528 29262 tablet_bootstrap.cc:492] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.680135 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.680657 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.681085 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.681854 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.003s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":700,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1691,"lbm_read_time_us":214,"lbm_reads_lt_1ms":12,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260809 07:58:46.682701 29085 raft_consensus.cc:2272] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.683486 29262 tablet_bootstrap.cc:654] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.684037 29085 ts_tablet_manager.cc:1916] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.691766 29085 ts_tablet_manager.cc:1929] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.694857 29257 ts_tablet_manager.cc:1929] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.695027 29257 log.cc:1199] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/550838d288be4ecf8a9dd10aeda6467a
I20260809 07:58:46.695451 29257 ts_tablet_manager.cc:1950] T 550838d288be4ecf8a9dd10aeda6467a P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.696058 29263 tablet_bootstrap.cc:492] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.696246 29263 ts_tablet_manager.cc:1403] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.040s user 0.012s sys 0.014s
I20260809 07:58:46.696573 29257 tablet_replica.cc:333] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.696717 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 550838d288be4ecf8a9dd10aeda6467a (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.696810 29257 raft_consensus.cc:2243] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.697229 29257 raft_consensus.cc:2272] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.697633 29263 raft_consensus.cc:359] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.697906 29263 raft_consensus.cc:385] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.698001 29263 raft_consensus.cc:740] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.698143 29257 ts_tablet_manager.cc:1916] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.698240 29260 tablet_bootstrap.cc:492] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.698267 29263 consensus_queue.cc:260] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.698442 29085 log.cc:1199] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/ab1a97f98f5446daa2523cf10fb8fac4
I20260809 07:58:46.698629 29260 ts_tablet_manager.cc:1403] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.033s user 0.012s sys 0.011s
I20260809 07:58:46.698956 29085 ts_tablet_manager.cc:1950] T ab1a97f98f5446daa2523cf10fb8fac4 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.699671 29260 raft_consensus.cc:359] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.699940 29260 raft_consensus.cc:385] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.700116 29260 raft_consensus.cc:740] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.700177 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet ab1a97f98f5446daa2523cf10fb8fac4 (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.700480 29260 consensus_queue.cc:260] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.700628 29262 tablet_bootstrap.cc:492] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.700801 29262 ts_tablet_manager.cc:1403] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.005s
I20260809 07:58:46.701680 29262 raft_consensus.cc:359] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.701903 29262 raft_consensus.cc:385] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.702018 29260 ts_tablet_manager.cc:1434] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:46.702001 29262 raft_consensus.cc:740] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.702276 29262 consensus_queue.cc:260] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.703724 29262 ts_tablet_manager.cc:1434] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:46.703933 29260 tablet_bootstrap.cc:492] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.704057 29262 tablet_bootstrap.cc:492] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.704679 29257 ts_tablet_manager.cc:1929] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.704823 29257 log.cc:1199] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/a48541470e51457d980a8c3ea27b060f
I20260809 07:58:46.705296 29257 ts_tablet_manager.cc:1950] T a48541470e51457d980a8c3ea27b060f P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.706115 29260 tablet_bootstrap.cc:654] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.706300 29257 tablet_replica.cc:333] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:46.706399 29262 tablet_bootstrap.cc:654] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.706525 29257 raft_consensus.cc:2243] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:46.706743 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet a48541470e51457d980a8c3ea27b060f (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.706861 29257 raft_consensus.cc:2272] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.707685 29257 ts_tablet_manager.cc:1916] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.707986 29014 raft_consensus.cc:493] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.708210 29014 raft_consensus.cc:515] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.708606 29085 tablet_replica.cc:333] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.708844 29014 leader_election.cc:290] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.709729 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e7b2737e49c41438b6478656c6df0a6" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.709954 28648 raft_consensus.cc:2468] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:46.710731 28380 leader_election.cc:304] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.710762 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e7b2737e49c41438b6478656c6df0a6" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:46.711037 28826 raft_consensus.cc:2468] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:46.713833 29014 raft_consensus.cc:2804] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.714017 29014 raft_consensus.cc:493] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.714248 29014 raft_consensus.cc:3060] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.717530 29257 ts_tablet_manager.cc:1929] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.717680 29257 log.cc:1199] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/eff7ca7c344b48b685c829148b184b9b
I20260809 07:58:46.718132 29257 ts_tablet_manager.cc:1950] T eff7ca7c344b48b685c829148b184b9b P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:46.719187 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet eff7ca7c344b48b685c829148b184b9b (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.719609 29014 raft_consensus.cc:515] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.720358 29014 leader_election.cc:290] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.720862 29263 ts_tablet_manager.cc:1434] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20260809 07:58:46.721252 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e7b2737e49c41438b6478656c6df0a6" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:46.721472 28648 raft_consensus.cc:3060] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.722223 29263 tablet_bootstrap.cc:492] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.723810 29263 tablet_bootstrap.cc:654] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.708868 29085 raft_consensus.cc:2243] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.724385 28648 raft_consensus.cc:2468] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:46.724475 29085 raft_consensus.cc:2272] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.726994 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e7b2737e49c41438b6478656c6df0a6" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:46.727268 29085 ts_tablet_manager.cc:1916] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.727289 28826 raft_consensus.cc:3060] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.728020 28380 leader_election.cc:304] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.728287 29014 raft_consensus.cc:2804] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.731124 28826 raft_consensus.cc:2468] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:46.738461 29262 tablet_bootstrap.cc:492] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.738631 29262 ts_tablet_manager.cc:1403] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.035s user 0.012s sys 0.014s
I20260809 07:58:46.739207 29085 ts_tablet_manager.cc:1929] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.739437 29085 log.cc:1199] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/550838d288be4ecf8a9dd10aeda6467a
I20260809 07:58:46.739598 29262 raft_consensus.cc:359] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.739979 29085 ts_tablet_manager.cc:1950] T 550838d288be4ecf8a9dd10aeda6467a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.739997 29262 raft_consensus.cc:385] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.740116 29262 raft_consensus.cc:740] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.740463 29262 consensus_queue.cc:260] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.740842 29014 raft_consensus.cc:697] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:46.741140 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 550838d288be4ecf8a9dd10aeda6467a (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.741138 29014 consensus_queue.cc:237] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.742779 29085 tablet_replica.cc:333] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.742972 29085 raft_consensus.cc:2243] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.743198 29085 raft_consensus.cc:2272] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.743886 29262 ts_tablet_manager.cc:1434] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:58:46.744468 29085 ts_tablet_manager.cc:1916] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.744874 29262 tablet_bootstrap.cc:492] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.748095 29263 tablet_bootstrap.cc:492] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.748257 29263 ts_tablet_manager.cc:1403] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.002s
I20260809 07:58:46.748984 29263 raft_consensus.cc:359] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.749172 29263 raft_consensus.cc:385] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.749286 29263 raft_consensus.cc:740] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.749557 29263 consensus_queue.cc:260] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.750135 29263 ts_tablet_manager.cc:1434] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:46.750469 29262 tablet_bootstrap.cc:654] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.751711 29263 tablet_bootstrap.cc:492] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.753578 29263 tablet_bootstrap.cc:654] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.753429 28149 catalog_manager.cc:5719] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:46.763345 29085 ts_tablet_manager.cc:1929] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.764333 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.764673 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.765030 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.765476 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.765846 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.766008 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.766206 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.766255 29085 log.cc:1199] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/a48541470e51457d980a8c3ea27b060f
I20260809 07:58:46.766302 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.766506 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.766742 29085 ts_tablet_manager.cc:1950] T a48541470e51457d980a8c3ea27b060f P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.766812 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.767104 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.768575 29260 tablet_bootstrap.cc:492] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.768782 29260 ts_tablet_manager.cc:1403] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.065s user 0.020s sys 0.020s
I20260809 07:58:46.768853 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.769169 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.769474 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.769811 29260 raft_consensus.cc:359] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.769976 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.770068 29260 raft_consensus.cc:385] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.770176 29260 raft_consensus.cc:740] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.770299 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.770476 29260 consensus_queue.cc:260] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.770593 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.770887 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.771086 29260 ts_tablet_manager.cc:1434] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:46.771148 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.765494 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.772680 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet a48541470e51457d980a8c3ea27b060f (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.772940 29260 tablet_bootstrap.cc:492] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.776587 29260 tablet_bootstrap.cc:654] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.777884 29085 tablet_replica.cc:333] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:46.778115 29085 raft_consensus.cc:2243] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:46.778326 29085 raft_consensus.cc:2272] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:46.781049 29263 tablet_bootstrap.cc:492] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.781199 29085 ts_tablet_manager.cc:1916] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:46.781220 29263 ts_tablet_manager.cc:1403] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.018s
I20260809 07:58:46.782025 29263 raft_consensus.cc:359] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.782244 29263 raft_consensus.cc:385] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.782320 29263 raft_consensus.cc:740] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.782606 29263 consensus_queue.cc:260] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.783635 29262 tablet_bootstrap.cc:492] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.783900 29262 ts_tablet_manager.cc:1403] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.039s user 0.014s sys 0.009s
I20260809 07:58:46.784076 29263 ts_tablet_manager.cc:1434] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:46.784389 29263 tablet_bootstrap.cc:492] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.785017 29262 raft_consensus.cc:359] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.785358 29262 raft_consensus.cc:385] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.785522 29262 raft_consensus.cc:740] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.785914 29262 consensus_queue.cc:260] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.786226 29263 tablet_bootstrap.cc:654] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.786696 29262 ts_tablet_manager.cc:1434] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:46.787613 29262 tablet_bootstrap.cc:492] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.788262 29085 ts_tablet_manager.cc:1929] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:46.788388 29085 log.cc:1199] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/eff7ca7c344b48b685c829148b184b9b
I20260809 07:58:46.789026 29085 ts_tablet_manager.cc:1950] T eff7ca7c344b48b685c829148b184b9b P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:46.790351 29262 tablet_bootstrap.cc:654] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.791849 29260 tablet_bootstrap.cc:492] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.792002 29260 ts_tablet_manager.cc:1403] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.003s
I20260809 07:58:46.792807 29260 raft_consensus.cc:359] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.792994 29260 raft_consensus.cc:385] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.793095 29260 raft_consensus.cc:740] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.793355 29260 consensus_queue.cc:260] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.793926 29260 ts_tablet_manager.cc:1434] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:46.794301 29260 tablet_bootstrap.cc:492] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.794588 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet eff7ca7c344b48b685c829148b184b9b (table table-1 [id=02e8fe507dcb47419972879b80716181]) successfully deleted
I20260809 07:58:46.796089 29260 tablet_bootstrap.cc:654] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.796998 29262 tablet_bootstrap.cc:492] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.797240 29262 ts_tablet_manager.cc:1403] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.003s
I20260809 07:58:46.798321 29262 raft_consensus.cc:359] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.798691 29262 raft_consensus.cc:385] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.798864 29262 raft_consensus.cc:740] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.799253 29262 consensus_queue.cc:260] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.800680 29262 ts_tablet_manager.cc:1434] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:46.801126 29262 tablet_bootstrap.cc:492] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.804471 29262 tablet_bootstrap.cc:654] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.809091 29260 tablet_bootstrap.cc:492] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.809301 29260 ts_tablet_manager.cc:1403] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.012s
I20260809 07:58:46.810077 29260 raft_consensus.cc:359] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.810266 29260 raft_consensus.cc:385] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.810346 29260 raft_consensus.cc:740] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.810931 29260 consensus_queue.cc:260] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.811952 29260 ts_tablet_manager.cc:1434] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260809 07:58:46.812260 29260 tablet_bootstrap.cc:492] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.814723 29014 raft_consensus.cc:493] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.814870 29014 raft_consensus.cc:515] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.815485 29014 leader_election.cc:290] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.816461 29260 tablet_bootstrap.cc:654] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.816583 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7592e4276d0d4ddbb0d59cfec840984d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.816874 28648 raft_consensus.cc:2468] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:46.817589 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7592e4276d0d4ddbb0d59cfec840984d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:46.817647 28380 leader_election.cc:304] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.817911 29014 raft_consensus.cc:2804] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.818040 29014 raft_consensus.cc:493] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.818158 29014 raft_consensus.cc:3060] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:58:46.818228 28382 leader_election.cc:343] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.822299 29014 raft_consensus.cc:515] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.822808 29262 tablet_bootstrap.cc:492] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.822980 29262 ts_tablet_manager.cc:1403] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.007s
I20260809 07:58:46.823151 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.058s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.823750 29262 raft_consensus.cc:359] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.823945 29262 raft_consensus.cc:385] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.824033 29262 raft_consensus.cc:740] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.824278 29262 consensus_queue.cc:260] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.824941 29262 ts_tablet_manager.cc:1434] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:46.825124 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7592e4276d0d4ddbb0d59cfec840984d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:46.825251 29262 tablet_bootstrap.cc:492] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.825359 28648 raft_consensus.cc:3060] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.825420 29014 leader_election.cc:290] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.825673 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7592e4276d0d4ddbb0d59cfec840984d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
W20260809 07:58:46.826279 28382 leader_election.cc:343] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.827275 29262 tablet_bootstrap.cc:654] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.828437 28648 raft_consensus.cc:2468] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:46.828898 28380 leader_election.cc:304] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:46.829216 29014 raft_consensus.cc:2804] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.829384 29014 raft_consensus.cc:697] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:46.829672 29014 consensus_queue.cc:237] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.833877 29211 raft_consensus.cc:493] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.834059 29211 raft_consensus.cc:515] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.834796 29211 leader_election.cc:290] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:46.835037 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.068s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.835678 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bf1f8cb2d2a43668e0aad3ef4052be6" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.835933 28461 raft_consensus.cc:2468] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:46.836364 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bf1f8cb2d2a43668e0aad3ef4052be6" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.836601 28648 raft_consensus.cc:2468] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:46.837219 28740 leader_election.cc:304] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:46.837565 29211 raft_consensus.cc:2804] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.837673 29211 raft_consensus.cc:493] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.837771 29211 raft_consensus.cc:3060] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.838251 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.067s user 0.007s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":727,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":65,"lbm_read_time_us":179,"lbm_reads_lt_1ms":8,"lbm_write_time_us":153,"lbm_writes_lt_1ms":4,"mutex_wait_us":7004,"reinsert_count":0,"update_count":4}
I20260809 07:58:46.839298 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.073s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":727,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1537,"lbm_read_time_us":157,"lbm_reads_lt_1ms":8,"lbm_write_time_us":223,"lbm_writes_lt_1ms":4,"mutex_wait_us":47904,"reinsert_count":0,"update_count":4}
I20260809 07:58:46.841277 29211 raft_consensus.cc:515] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.842065 29211 leader_election.cc:290] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:46.842754 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bf1f8cb2d2a43668e0aad3ef4052be6" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:46.842932 28461 raft_consensus.cc:3060] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.843055 29262 tablet_bootstrap.cc:492] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.843189 29262 ts_tablet_manager.cc:1403] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260809 07:58:46.843211 29260 tablet_bootstrap.cc:492] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.843343 29260 ts_tablet_manager.cc:1403] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.013s
I20260809 07:58:46.844053 29262 raft_consensus.cc:359] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.844144 29260 raft_consensus.cc:359] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.844256 29262 raft_consensus.cc:385] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.844324 29260 raft_consensus.cc:385] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.844341 29262 raft_consensus.cc:740] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.844408 29260 raft_consensus.cc:740] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.844605 29262 consensus_queue.cc:260] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.844703 29260 consensus_queue.cc:260] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.845146 29262 ts_tablet_manager.cc:1434] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:46.845227 29260 ts_tablet_manager.cc:1434] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:46.845528 29260 tablet_bootstrap.cc:492] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.845955 29262 tablet_bootstrap.cc:492] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.846079 28461 raft_consensus.cc:2468] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:46.846171 29263 tablet_bootstrap.cc:492] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.846310 29263 ts_tablet_manager.cc:1403] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.062s user 0.023s sys 0.019s
I20260809 07:58:46.846866 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bf1f8cb2d2a43668e0aad3ef4052be6" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:46.847038 28648 raft_consensus.cc:3060] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.847436 29263 raft_consensus.cc:359] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.847630 29263 raft_consensus.cc:385] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.847733 29263 raft_consensus.cc:740] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.847983 29263 consensus_queue.cc:260] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.848879 29260 tablet_bootstrap.cc:654] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.849561 29263 ts_tablet_manager.cc:1434] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:46.850014 29263 tablet_bootstrap.cc:492] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.851264 28740 leader_election.cc:304] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:46.851548 29211 raft_consensus.cc:2804] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.851691 29211 raft_consensus.cc:697] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:46.852016 29211 consensus_queue.cc:237] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.853318 29262 tablet_bootstrap.cc:654] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.855055 28648 raft_consensus.cc:2468] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:46.855392 28961 raft_consensus.cc:493] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.855661 28961 raft_consensus.cc:515] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.856305 28961 leader_election.cc:290] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.856869 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba22ea1e78504b34ba032ce8575f0890" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.857409 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba22ea1e78504b34ba032ce8575f0890" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:46.857445 28565 leader_election.cc:343] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:46.857916 28566 leader_election.cc:343] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:46.858045 28566 leader_election.cc:304] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:46.858518 28961 raft_consensus.cc:2749] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:46.859274 28153 catalog_manager.cc:5719] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:46.860334 29263 tablet_bootstrap.cc:654] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.865062 29262 tablet_bootstrap.cc:492] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.865203 29262 ts_tablet_manager.cc:1403] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.003s
I20260809 07:58:46.866811 28151 catalog_manager.cc:5719] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:46.867658 29014 raft_consensus.cc:493] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.867853 29014 raft_consensus.cc:515] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.868558 29014 leader_election.cc:290] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.868810 29262 raft_consensus.cc:359] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.869009 29262 raft_consensus.cc:385] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.869091 29262 raft_consensus.cc:740] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.869410 29262 consensus_queue.cc:260] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.870245 29263 tablet_bootstrap.cc:492] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.870512 29263 ts_tablet_manager.cc:1403] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.001s
I20260809 07:58:46.871411 29263 raft_consensus.cc:359] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.871622 29263 raft_consensus.cc:385] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.871718 29263 raft_consensus.cc:740] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.871797 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f152550cc1194a04962055b95dee8e78" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.872017 28648 raft_consensus.cc:2468] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:46.872049 29263 consensus_queue.cc:260] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.872326 29260 tablet_bootstrap.cc:492] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.872543 28380 leader_election.cc:304] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.873142 29263 ts_tablet_manager.cc:1434] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:46.873315 29262 ts_tablet_manager.cc:1434] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260809 07:58:46.873508 29263 tablet_bootstrap.cc:492] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.874177 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f152550cc1194a04962055b95dee8e78" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:46.874460 28826 raft_consensus.cc:2468] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:46.874796 29014 raft_consensus.cc:2804] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.874887 29014 raft_consensus.cc:493] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.874984 29014 raft_consensus.cc:3060] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.875325 29263 tablet_bootstrap.cc:654] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.877154 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.877758 28961 raft_consensus.cc:493] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.877930 28961 raft_consensus.cc:515] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.878532 28961 leader_election.cc:290] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.878746 29014 raft_consensus.cc:515] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.877599 29262 tablet_bootstrap.cc:492] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.879527 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744b57379dd54528a734afcedf8128b7" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:46.879694 28461 raft_consensus.cc:2468] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:46.880026 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f152550cc1194a04962055b95dee8e78" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:46.880318 28648 raft_consensus.cc:3060] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.880519 28565 leader_election.cc:304] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.880986 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.881094 29262 tablet_bootstrap.cc:654] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.881475 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.884423 28648 raft_consensus.cc:2468] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:46.884755 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.884974 28380 leader_election.cc:304] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.885061 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.885291 29314 raft_consensus.cc:2804] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.885726 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.884429 28961 raft_consensus.cc:2804] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:46.886920 28961 raft_consensus.cc:493] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:46.887058 28961 raft_consensus.cc:3060] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.889868 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744b57379dd54528a734afcedf8128b7" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:46.890157 28826 raft_consensus.cc:2468] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:46.891647 29014 leader_election.cc:290] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.891992 29260 ts_tablet_manager.cc:1403] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.004s
I20260809 07:58:46.893576 29260 raft_consensus.cc:359] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.893841 29260 raft_consensus.cc:385] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.893932 29260 raft_consensus.cc:740] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.894227 29260 consensus_queue.cc:260] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.894984 29260 ts_tablet_manager.cc:1434] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:46.895294 29260 tablet_bootstrap.cc:492] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.898579 29260 tablet_bootstrap.cc:654] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.899219 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.899554 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.899857 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.900141 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.903352 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.903668 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.903939 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.906214 29314 raft_consensus.cc:697] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:46.906613 29314 consensus_queue.cc:237] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.910310 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.119528
I20260809 07:58:46.912071 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:46.912474 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.912711 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.915076 29340 raft_consensus.cc:493] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:46.915270 29340 raft_consensus.cc:515] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.916249 29340 leader_election.cc:290] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.917320 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dce89653b584c46a1128e0c9d43356d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:46.917635 28648 raft_consensus.cc:2393] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in current term 1: Already voted for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in this term.
I20260809 07:58:46.918392 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dce89653b584c46a1128e0c9d43356d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:46.918905 29260 tablet_bootstrap.cc:492] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.919034 29260 ts_tablet_manager.cc:1403] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.005s
I20260809 07:58:46.919086 29211 consensus_queue.cc:1048] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.919346 28382 leader_election.cc:304] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:46.919909 29260 raft_consensus.cc:359] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.920089 29260 raft_consensus.cc:385] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.920190 29260 raft_consensus.cc:740] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.920423 29260 consensus_queue.cc:260] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.920804 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f152550cc1194a04962055b95dee8e78" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:46.921021 28826 raft_consensus.cc:3060] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.921790 28151 catalog_manager.cc:5719] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:46.922019 29014 raft_consensus.cc:3060] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.925254 29014 raft_consensus.cc:2749] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:46.927264 29260 ts_tablet_manager.cc:1434] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260809 07:58:46.928483 28826 raft_consensus.cc:2468] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:46.910629 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.119528
I20260809 07:58:46.929867 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.930197 29260 tablet_bootstrap.cc:492] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.931979 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.932235 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.003s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:46.933256 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.933512 29260 tablet_bootstrap.cc:654] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.934473 29211 consensus_queue.cc:1048] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.938562 28961 raft_consensus.cc:515] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.939424 28961 leader_election.cc:290] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:46.939657 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.062s user 0.001s sys 0.004s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":179,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:46.942339 29260 tablet_bootstrap.cc:492] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.942588 29260 ts_tablet_manager.cc:1403] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.005s
I20260809 07:58:46.943125 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.006s sys 0.000s Metrics: {"bytes_written":465,"delete_count":0,"lbm_write_time_us":200,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 07:58:46.943771 29260 raft_consensus.cc:359] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.944023 29260 raft_consensus.cc:385] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.944131 29260 raft_consensus.cc:740] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.944425 29260 consensus_queue.cc:260] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.942268 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744b57379dd54528a734afcedf8128b7" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:46.945511 28464 raft_consensus.cc:3060] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:46.945787 29260 ts_tablet_manager.cc:1434] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:46.946086 29260 tablet_bootstrap.cc:492] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.947851 29260 tablet_bootstrap.cc:654] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.959919 29263 tablet_bootstrap.cc:492] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:46.960081 29263 ts_tablet_manager.cc:1403] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.087s user 0.016s sys 0.022s
I20260809 07:58:46.960193 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.960548 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:46.961503 28464 raft_consensus.cc:2468] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:46.962388 28565 leader_election.cc:304] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:46.962775 28961 raft_consensus.cc:2804] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:46.962944 28961 raft_consensus.cc:697] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:46.963230 28961 consensus_queue.cc:237] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.964756 29029 consensus_queue.cc:1048] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.965092 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.080s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":752,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":201,"lbm_reads_lt_1ms":8,"lbm_write_time_us":191,"lbm_writes_lt_1ms":4,"mutex_wait_us":20134,"reinsert_count":0,"update_count":5}
I20260809 07:58:46.965821 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121102
I20260809 07:58:46.968804 29262 tablet_bootstrap.cc:492] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:46.968952 29262 ts_tablet_manager.cc:1403] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.091s user 0.039s sys 0.024s
I20260809 07:58:46.969717 29262 raft_consensus.cc:359] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.969864 29262 raft_consensus.cc:385] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.969942 29262 raft_consensus.cc:740] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.970127 29262 consensus_queue.cc:260] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.970620 29262 ts_tablet_manager.cc:1434] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:46.970911 29262 tablet_bootstrap.cc:492] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:46.971614 29029 consensus_queue.cc:1048] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:46.972697 29262 tablet_bootstrap.cc:654] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.972980 29263 raft_consensus.cc:359] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.973196 29263 raft_consensus.cc:385] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.973299 29263 raft_consensus.cc:740] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.973685 29263 consensus_queue.cc:260] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.975996 29263 ts_tablet_manager.cc:1434] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260809 07:58:46.976296 29263 tablet_bootstrap.cc:492] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:46.978907 29263 tablet_bootstrap.cc:654] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.985991 29260 tablet_bootstrap.cc:492] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:46.986131 29260 ts_tablet_manager.cc:1403] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.002s
I20260809 07:58:46.986872 29260 raft_consensus.cc:359] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.987098 29260 raft_consensus.cc:385] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:46.987211 29260 raft_consensus.cc:740] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:46.987432 29260 consensus_queue.cc:260] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:46.988153 29260 ts_tablet_manager.cc:1434] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:46.988533 29260 tablet_bootstrap.cc:492] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:46.991123 29260 tablet_bootstrap.cc:654] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:46.994735 28151 catalog_manager.cc:5719] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:46.997140 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.067s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":66869}
I20260809 07:58:46.997541 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":36817}
I20260809 07:58:47.010756 29314 raft_consensus.cc:493] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.010974 29314 raft_consensus.cc:515] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.011714 29314 leader_election.cc:290] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.011962 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744b57379dd54528a734afcedf8128b7" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:47.013513 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a2e99f189940be8e9fb9981c740468" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:47.013756 28649 raft_consensus.cc:2468] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:47.014253 28380 leader_election.cc:304] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.014573 29314 raft_consensus.cc:2804] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:47.014686 29314 raft_consensus.cc:493] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:47.014803 29314 raft_consensus.cc:3060] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.017781 29314 raft_consensus.cc:515] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.018922 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a2e99f189940be8e9fb9981c740468" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:47.019131 28649 raft_consensus.cc:3060] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.019239 29314 leader_election.cc:290] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.020323 29260 tablet_bootstrap.cc:492] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:47.020478 29260 ts_tablet_manager.cc:1403] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.009s
I20260809 07:58:47.021236 29260 raft_consensus.cc:359] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.021467 29260 raft_consensus.cc:385] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.021548 29260 raft_consensus.cc:740] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.021855 29260 consensus_queue.cc:260] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.022924 28649 raft_consensus.cc:2468] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:47.023329 28380 leader_election.cc:304] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.023815 29314 raft_consensus.cc:2804] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:47.023996 29314 raft_consensus.cc:697] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:47.024253 29314 consensus_queue.cc:237] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.028505 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.028847 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.030704 29260 ts_tablet_manager.cc:1434] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260809 07:58:47.030884 28961 consensus_queue.cc:1048] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.033236 28149 catalog_manager.cc:5719] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:47.039420 28961 raft_consensus.cc:493] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.039585 28961 raft_consensus.cc:515] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.040905 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafaca6dde5b412e83e9f4cc1edb1f66" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:47.041102 28464 raft_consensus.cc:2468] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:47.041621 28565 leader_election.cc:304] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.041885 29029 raft_consensus.cc:2804] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:47.042019 29029 raft_consensus.cc:493] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:47.042141 29029 raft_consensus.cc:3060] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.042181 28961 leader_election.cc:290] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.045152 29029 raft_consensus.cc:515] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.045954 29029 leader_election.cc:290] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.029784 29211 consensus_queue.cc:1048] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.046475 28826 raft_consensus.cc:3060] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.047940 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafaca6dde5b412e83e9f4cc1edb1f66" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:47.048130 28464 raft_consensus.cc:3060] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.051011 28464 raft_consensus.cc:2468] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:47.051461 28565 leader_election.cc:304] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.051838 29029 raft_consensus.cc:2804] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:47.052052 29029 raft_consensus.cc:697] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:47.052909 29029 consensus_queue.cc:237] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.059669 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a2e99f189940be8e9fb9981c740468" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:47.056097 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.143s user 0.006s sys 0.000s Metrics: {"bytes_written":465,"delete_count":0,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"mutex_wait_us":52382,"reinsert_count":0,"update_count":2}
I20260809 07:58:47.059921 28825 raft_consensus.cc:2468] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:47.060351 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.061127 28825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a2e99f189940be8e9fb9981c740468" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:47.061450 28825 raft_consensus.cc:3060] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.061664 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafaca6dde5b412e83e9f4cc1edb1f66" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:47.061949 28819 raft_consensus.cc:3060] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.064603 28825 raft_consensus.cc:2468] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:47.064949 28819 raft_consensus.cc:2468] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:47.066330 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafaca6dde5b412e83e9f4cc1edb1f66" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:47.066699 28820 raft_consensus.cc:2376] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1. Re-sending same reply.
I20260809 07:58:47.067875 29211 consensus_queue.cc:1048] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.071950 29198 consensus_queue.cc:1048] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.074666 28153 catalog_manager.cc:5719] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:47.079663 28826 raft_consensus.cc:2468] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:47.089028 28941 consensus_queue.cc:1048] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.105486 28941 consensus_queue.cc:1048] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.120739 29001 raft_consensus.cc:493] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.120920 29001 raft_consensus.cc:515] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.121747 29001 leader_election.cc:290] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:47.127557 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b65040b5e344742890ad101393efc48" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:47.127869 28464 raft_consensus.cc:2468] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:47.128683 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b65040b5e344742890ad101393efc48" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:47.128904 28648 raft_consensus.cc:2468] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:47.136448 28740 leader_election.cc:304] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:47.137782 28941 raft_consensus.cc:2804] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:47.137918 28941 raft_consensus.cc:493] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:47.138010 28941 raft_consensus.cc:3060] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.145524 29314 raft_consensus.cc:493] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.145679 29314 raft_consensus.cc:515] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.146453 29314 leader_election.cc:290] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.147567 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.236s user 0.014s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":17684,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":3182,"dirs.run_cpu_time_us":308,"dirs.run_wall_time_us":1513,"lbm_read_time_us":14338,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":26,"lbm_write_time_us":459,"lbm_writes_lt_1ms":9,"mutex_wait_us":31204,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":14014,"threads_started":1,"update_count":6}
I20260809 07:58:47.150108 29263 tablet_bootstrap.cc:492] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:47.150298 29263 ts_tablet_manager.cc:1403] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.174s user 0.030s sys 0.029s
I20260809 07:58:47.151031 29263 raft_consensus.cc:359] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.151221 29263 raft_consensus.cc:385] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.151286 29263 raft_consensus.cc:740] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.151495 29263 consensus_queue.cc:260] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.151978 29263 ts_tablet_manager.cc:1434] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:47.152281 29263 tablet_bootstrap.cc:492] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:47.152678 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.219s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":219061}
I20260809 07:58:47.152944 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.192s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":192069}
I20260809 07:58:47.153640 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.221s user 0.005s sys 0.000s Metrics: {"bytes_written":510,"delete_count":0,"lbm_write_time_us":149,"lbm_writes_lt_1ms":4,"mutex_wait_us":215017,"reinsert_count":0,"update_count":4}
I20260809 07:58:47.154052 29262 tablet_bootstrap.cc:492] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:47.154233 29262 ts_tablet_manager.cc:1403] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.183s user 0.048s sys 0.093s
I20260809 07:58:47.154518 28941 raft_consensus.cc:515] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.155074 29262 raft_consensus.cc:359] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.155282 29262 raft_consensus.cc:385] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.155284 28941 leader_election.cc:290] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:47.155365 29262 raft_consensus.cc:740] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.155606 29262 consensus_queue.cc:260] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.156246 29262 ts_tablet_manager.cc:1434] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:47.156297 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.127s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":126870}
I20260809 07:58:47.156584 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.095s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":94674}
I20260809 07:58:47.157745 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.129s user 0.000s sys 0.005s Metrics: {"bytes_written":487,"delete_count":0,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"mutex_wait_us":106836,"reinsert_count":0,"update_count":3}
I20260809 07:58:47.158078 29263 tablet_bootstrap.cc:654] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.159155 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad6dd9c01dd45f3a16a9be616205018" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:47.159377 28648 raft_consensus.cc:2468] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:47.159825 28380 leader_election.cc:304] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.160180 29314 raft_consensus.cc:2804] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:47.160295 29314 raft_consensus.cc:493] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:47.160380 29314 raft_consensus.cc:3060] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.164330 29314 raft_consensus.cc:515] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.165964 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad6dd9c01dd45f3a16a9be616205018" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:47.166148 28649 raft_consensus.cc:3060] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.167469 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b65040b5e344742890ad101393efc48" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:47.167639 28461 raft_consensus.cc:3060] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.169143 28649 raft_consensus.cc:2468] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:47.170773 28461 raft_consensus.cc:2468] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:47.170809 28380 leader_election.cc:304] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.171118 29340 raft_consensus.cc:2804] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:47.171291 29340 raft_consensus.cc:697] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:47.171545 29340 consensus_queue.cc:237] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.172811 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b65040b5e344742890ad101393efc48" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:47.173018 28649 raft_consensus.cc:3060] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.173867 28740 leader_election.cc:304] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:47.174265 28941 raft_consensus.cc:2804] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:47.175417 28941 raft_consensus.cc:697] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:47.175915 28941 consensus_queue.cc:237] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.176249 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad6dd9c01dd45f3a16a9be616205018" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:47.178085 28382 leader_election.cc:343] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:47.180734 28153 catalog_manager.cc:5719] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:47.182102 28649 raft_consensus.cc:2468] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:47.182879 29314 leader_election.cc:290] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.185302 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.219s user 0.014s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":17714,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":29317,"dirs.run_cpu_time_us":280,"dirs.run_wall_time_us":1054,"lbm_read_time_us":674,"lbm_reads_lt_1ms":27,"lbm_write_time_us":413,"lbm_writes_lt_1ms":9,"mutex_wait_us":31235,"peak_mem_usage":32768,"reinsert_count":0,"update_count":7}
I20260809 07:58:47.190897 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad6dd9c01dd45f3a16a9be616205018" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:47.191355 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.191622 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.191849 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.192137 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.193553 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.193751 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.193912 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.194308 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.194563 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.194788 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.195143 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.195329 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.198048 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.198395 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.198714 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.199033 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
W20260809 07:58:47.199966 28382 leader_election.cc:343] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:47.200323 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.200567 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.200737 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.202533 28153 catalog_manager.cc:5719] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:47.205026 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.205307 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.205775 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.205793 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.004s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":135,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.206907 29263 tablet_bootstrap.cc:492] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:47.207185 29263 ts_tablet_manager.cc:1403] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.055s user 0.018s sys 0.006s
I20260809 07:58:47.207998 29263 raft_consensus.cc:359] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.208123 29263 raft_consensus.cc:385] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.208187 29263 raft_consensus.cc:740] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.208474 29263 consensus_queue.cc:260] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.209560 29263 ts_tablet_manager.cc:1434] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:47.209872 29263 tablet_bootstrap.cc:492] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:47.213119 29263 tablet_bootstrap.cc:654] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.216437 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.216699 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.216758 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.204977 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":123,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.217007 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.217319 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.217790 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.217852 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":744,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":2328,"lbm_read_time_us":123,"lbm_reads_lt_1ms":8,"lbm_write_time_us":139,"lbm_writes_lt_1ms":4,"mutex_wait_us":13312,"reinsert_count":0,"update_count":5}
I20260809 07:58:47.218314 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.218586 29314 raft_consensus.cc:493] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.218703 29314 raft_consensus.cc:515] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.219374 29314 leader_election.cc:290] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.220688 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":721,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":58,"lbm_read_time_us":186,"lbm_reads_lt_1ms":8,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":4}
I20260809 07:58:47.223505 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":5856}
I20260809 07:58:47.224295 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":6609}
I20260809 07:58:47.224411 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba22ea1e78504b34ba032ce8575f0890" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:47.224623 28648 raft_consensus.cc:2468] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:47.225076 28380 leader_election.cc:304] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.225417 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba22ea1e78504b34ba032ce8575f0890" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:47.225585 29314 raft_consensus.cc:2804] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:47.225643 28826 raft_consensus.cc:2468] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:47.225754 29314 raft_consensus.cc:493] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:47.225845 29314 raft_consensus.cc:3060] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.228708 29314 raft_consensus.cc:515] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.229969 29314 leader_election.cc:290] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.230086 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba22ea1e78504b34ba032ce8575f0890" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:47.230121 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":144,"lbm_writes_lt_1ms":4,"mutex_wait_us":340,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.230293 28826 raft_consensus.cc:3060] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.230284 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba22ea1e78504b34ba032ce8575f0890" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:47.230556 28649 raft_consensus.cc:3060] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.232827 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.004s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":776,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":66,"lbm_read_time_us":211,"lbm_reads_lt_1ms":8,"lbm_write_time_us":157,"lbm_writes_lt_1ms":4,"mutex_wait_us":5770,"reinsert_count":0,"update_count":6}
I20260809 07:58:47.233330 28826 raft_consensus.cc:2468] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:47.233577 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.233865 28382 leader_election.cc:304] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:47.234184 29314 raft_consensus.cc:2804] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:47.235037 29314 raft_consensus.cc:697] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:47.235309 29314 consensus_queue.cc:237] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.235647 29340 consensus_queue.cc:1048] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.238206 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":128,"lbm_writes_lt_1ms":4,"mutex_wait_us":3240,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.239537 28941 raft_consensus.cc:493] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.239732 28941 raft_consensus.cc:515] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.240248 28941 leader_election.cc:290] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:47.240870 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7592e4276d0d4ddbb0d59cfec840984d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:47.240828 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7592e4276d0d4ddbb0d59cfec840984d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:47.241092 28648 raft_consensus.cc:2393] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a091b20dbc4ed2b3bf6c960df964c5 in current term 1: Already voted for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in this term.
I20260809 07:58:47.241750 28740 leader_election.cc:304] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:47.242036 28941 raft_consensus.cc:3060] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.238282 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":20456}
I20260809 07:58:47.238499 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":20415}
I20260809 07:58:47.242417 29263 tablet_bootstrap.cc:492] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:47.242599 29263 ts_tablet_manager.cc:1403] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.000s
I20260809 07:58:47.243410 29263 raft_consensus.cc:359] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.243625 28649 raft_consensus.cc:2468] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:47.243651 29263 raft_consensus.cc:385] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.243752 29263 raft_consensus.cc:740] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.244046 29263 consensus_queue.cc:260] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.244573 29263 ts_tablet_manager.cc:1434] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:47.245066 28941 raft_consensus.cc:2749] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:47.248538 29340 consensus_queue.cc:1048] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.252614 28151 catalog_manager.cc:5719] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:47.255187 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.005s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":743,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":146,"lbm_reads_lt_1ms":8,"lbm_write_time_us":123,"lbm_writes_lt_1ms":4,"mutex_wait_us":4636,"reinsert_count":0,"update_count":5}
I20260809 07:58:47.258507 29014 consensus_queue.cc:1048] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.263435 29014 consensus_queue.cc:1048] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.265770 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.266209 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.266590 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.267062 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.267395 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.267789 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.268173 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.268570 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.269526 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.269780 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.269981 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.270421 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.270720 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.271054 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.271351 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.273643 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":122,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.274392 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.274731 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.275703 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.276121 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":144,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.280150 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.004s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":807,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":76,"lbm_read_time_us":140,"lbm_reads_lt_1ms":8,"lbm_write_time_us":214,"lbm_writes_lt_1ms":4,"mutex_wait_us":509,"reinsert_count":0,"update_count":7}
I20260809 07:58:47.281963 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.003s sys 0.003s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":782,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":88,"lbm_read_time_us":156,"lbm_reads_lt_1ms":8,"lbm_write_time_us":172,"lbm_writes_lt_1ms":4,"mutex_wait_us":1352,"reinsert_count":0,"update_count":6}
I20260809 07:58:47.291725 28941 consensus_queue.cc:1048] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.296800 29372 consensus_queue.cc:1048] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.329873 29198 raft_consensus.cc:493] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.330016 29198 raft_consensus.cc:515] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.330679 29198 leader_election.cc:290] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.331064 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bad7eb37a784618b13b2263c0215b21" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:47.331131 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bad7eb37a784618b13b2263c0215b21" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:47.331307 28461 raft_consensus.cc:2468] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:47.331342 28826 raft_consensus.cc:2468] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:47.331737 28565 leader_election.cc:304] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.332011 29198 raft_consensus.cc:2804] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:47.332132 29198 raft_consensus.cc:493] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:47.332234 29198 raft_consensus.cc:3060] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.334211 29198 raft_consensus.cc:515] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.334702 29198 leader_election.cc:290] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.335094 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bad7eb37a784618b13b2263c0215b21" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:47.335156 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bad7eb37a784618b13b2263c0215b21" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:47.335261 28461 raft_consensus.cc:3060] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.335335 28826 raft_consensus.cc:3060] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.337265 28461 raft_consensus.cc:2468] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:47.337402 28826 raft_consensus.cc:2468] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:47.337724 28565 leader_election.cc:304] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.338007 29198 raft_consensus.cc:2804] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:47.338177 29198 raft_consensus.cc:697] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:47.338454 29198 consensus_queue.cc:237] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.341111 28151 catalog_manager.cc:5719] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:47.352233 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.352655 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.352964 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.353439 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.353829 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.354122 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.354143 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.354493 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.354835 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.355195 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.355469 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.355772 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.356166 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.356211 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.357554 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":164,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.358969 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.002s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":165,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.363605 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.003s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":828,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":65,"lbm_read_time_us":111,"lbm_reads_lt_1ms":8,"lbm_write_time_us":158,"lbm_writes_lt_1ms":4,"mutex_wait_us":3880,"reinsert_count":0,"update_count":8}
I20260809 07:58:47.365048 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":803,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":54,"lbm_read_time_us":165,"lbm_reads_lt_1ms":8,"lbm_write_time_us":167,"lbm_writes_lt_1ms":4,"mutex_wait_us":2965,"reinsert_count":0,"update_count":7}
I20260809 07:58:47.368626 29198 consensus_queue.cc:1048] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.373427 29198 consensus_queue.cc:1048] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.400054 28153 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51156:
table { table_name: "table-2" } modify_external_catalogs: true
I20260809 07:58:47.400276 28153 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51156:
table { table_name: "table-2" } modify_external_catalogs: true
I20260809 07:58:47.411345 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet edd67bd7518a43ec94443bc1ed822a57
I20260809 07:58:47.411999 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet e9e62dc7df994bc8ace03052264aa9a2
I20260809 07:58:47.412436 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet edd67bd7518a43ec94443bc1ed822a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.412586 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet c0862fc949b040b09e9dd945b1bc3e37
I20260809 07:58:47.412918 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet edd67bd7518a43ec94443bc1ed822a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.413009 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 0255a05235b3407d98a10bf04b0c8080
I20260809 07:58:47.413298 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet edd67bd7518a43ec94443bc1ed822a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.413509 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 4e7b2737e49c41438b6478656c6df0a6
I20260809 07:58:47.413566 28428 tablet_service.cc:1558] Processing DeleteTablet for tablet c0862fc949b040b09e9dd945b1bc3e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.413668 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet e9e62dc7df994bc8ace03052264aa9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.413867 28620 tablet_service.cc:1558] Processing DeleteTablet for tablet e9e62dc7df994bc8ace03052264aa9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.413972 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 7dce89653b584c46a1128e0c9d43356d
I20260809 07:58:47.414041 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet c0862fc949b040b09e9dd945b1bc3e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.414590 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet 0255a05235b3407d98a10bf04b0c8080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.414600 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 7bf1f8cb2d2a43668e0aad3ef4052be6
I20260809 07:58:47.414686 28428 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e7b2737e49c41438b6478656c6df0a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.415024 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet 7dce89653b584c46a1128e0c9d43356d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.415287 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet 0255a05235b3407d98a10bf04b0c8080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.415405 28619 tablet_service.cc:1558] Processing DeleteTablet for tablet 7dce89653b584c46a1128e0c9d43356d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.415467 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.415508 28620 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e7b2737e49c41438b6478656c6df0a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.415822 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.416158 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.416249 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.416605 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.416893 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.417546 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.433634
I20260809 07:58:47.417775 29393 tablet_replica.cc:333] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.417894 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.433634
I20260809 07:58:47.418195 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.433634
I20260809 07:58:47.418212 29393 raft_consensus.cc:2243] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.418563 29393 raft_consensus.cc:2272] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.418555 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.418814 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.419304 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.419721 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.420624 29393 ts_tablet_manager.cc:1916] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.420774 29340 consensus_queue.cc:1048] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.421669 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.422011 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.423216 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.423554 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.424916 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet c0862fc949b040b09e9dd945b1bc3e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.425163 28782 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e7b2737e49c41438b6478656c6df0a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.425350 28794 tablet_service.cc:1558] Processing DeleteTablet for tablet 7dce89653b584c46a1128e0c9d43356d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.425503 28795 tablet_service.cc:1558] Processing DeleteTablet for tablet 0255a05235b3407d98a10bf04b0c8080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.425661 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.425812 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet e9e62dc7df994bc8ace03052264aa9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.426213 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.426486 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.426709 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.426682 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.431509
I20260809 07:58:47.427165 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.431509
I20260809 07:58:47.427484 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.431509
I20260809 07:58:47.427896 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.428200 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.428539 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.428915 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.434065 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.434290 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 28a2e99f189940be8e9fb9981c740468
I20260809 07:58:47.435168 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 28a2e99f189940be8e9fb9981c740468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.435704 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 28a2e99f189940be8e9fb9981c740468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.436254 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet 28a2e99f189940be8e9fb9981c740468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.436496 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 7592e4276d0d4ddbb0d59cfec840984d
I20260809 07:58:47.437180 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 7592e4276d0d4ddbb0d59cfec840984d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.437664 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 7592e4276d0d4ddbb0d59cfec840984d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.438123 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet 7592e4276d0d4ddbb0d59cfec840984d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.438326 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet dafaca6dde5b412e83e9f4cc1edb1f66
I20260809 07:58:47.438957 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet dafaca6dde5b412e83e9f4cc1edb1f66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.439600 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet dafaca6dde5b412e83e9f4cc1edb1f66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.440093 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet dafaca6dde5b412e83e9f4cc1edb1f66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.440326 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet f152550cc1194a04962055b95dee8e78
I20260809 07:58:47.441104 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet f152550cc1194a04962055b95dee8e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.441175 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.005s sys 0.000s Metrics: {"bytes_written":946,"delete_count":0,"lbm_write_time_us":176,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260809 07:58:47.441538 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet f152550cc1194a04962055b95dee8e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.440943 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.005s sys 0.000s Metrics: {"bytes_written":946,"delete_count":0,"lbm_write_time_us":206,"lbm_writes_lt_1ms":4,"reinsert_count":0,"spinlock_wait_cycles":28032,"update_count":17}
I20260809 07:58:47.442085 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet f152550cc1194a04962055b95dee8e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.442243 29392 tablet_replica.cc:333] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.442354 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet ba22ea1e78504b34ba032ce8575f0890
I20260809 07:58:47.442587 29392 raft_consensus.cc:2243] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.442982 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet ba22ea1e78504b34ba032ce8575f0890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.443308 29392 raft_consensus.cc:2272] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.443492 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet ba22ea1e78504b34ba032ce8575f0890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.443981 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 744b57379dd54528a734afcedf8128b7
I20260809 07:58:47.444176 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet ba22ea1e78504b34ba032ce8575f0890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.444758 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 744b57379dd54528a734afcedf8128b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.444861 29393 ts_tablet_manager.cc:1929] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.445032 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 744b57379dd54528a734afcedf8128b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.445453 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 0bad7eb37a784618b13b2263c0215b21
I20260809 07:58:47.445724 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet 744b57379dd54528a734afcedf8128b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.446298 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 0bad7eb37a784618b13b2263c0215b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.446256 29393 log.cc:1199] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/edd67bd7518a43ec94443bc1ed822a57
I20260809 07:58:47.446808 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 0bad7eb37a784618b13b2263c0215b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.446889 29393 ts_tablet_manager.cc:1950] T edd67bd7518a43ec94443bc1ed822a57 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.447314 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 1b65040b5e344742890ad101393efc48
I20260809 07:58:47.447371 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet 0bad7eb37a784618b13b2263c0215b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.447687 29392 ts_tablet_manager.cc:1916] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.447903 29393 tablet_replica.cc:333] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.448062 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet edd67bd7518a43ec94443bc1ed822a57 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.448763 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b65040b5e344742890ad101393efc48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.415923 29394 tablet_replica.cc:333] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.449222 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 1ad6dd9c01dd45f3a16a9be616205018
I20260809 07:58:47.449486 29394 raft_consensus.cc:2243] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.449821 29394 raft_consensus.cc:2272] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.451220 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ad6dd9c01dd45f3a16a9be616205018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.447968 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b65040b5e344742890ad101393efc48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.450312 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ad6dd9c01dd45f3a16a9be616205018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:47.451781 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ad6dd9c01dd45f3a16a9be616205018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:47.452028 29393 raft_consensus.cc:2243] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.452158 29394 ts_tablet_manager.cc:1916] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.452260 29393 raft_consensus.cc:2272] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.453271 28151 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51156:
name: "table-3"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
I20260809 07:58:47.453441 29393 ts_tablet_manager.cc:1916] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 07:58:47.454201 28151 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:58:47.454416 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b65040b5e344742890ad101393efc48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:47 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:47.459758 29394 ts_tablet_manager.cc:1929] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.454661 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.012s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":18491,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":8719,"dirs.run_cpu_time_us":174,"dirs.run_wall_time_us":692,"lbm_read_time_us":585,"lbm_reads_lt_1ms":27,"lbm_write_time_us":389,"lbm_writes_lt_1ms":9,"mutex_wait_us":10954,"peak_mem_usage":32768,"reinsert_count":0,"update_count":25}
I20260809 07:58:47.459923 29393 ts_tablet_manager.cc:1929] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.460088 29393 log.cc:1199] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/c0862fc949b040b09e9dd945b1bc3e37
I20260809 07:58:47.460536 29393 ts_tablet_manager.cc:1950] T c0862fc949b040b09e9dd945b1bc3e37 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.460366 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.012s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":18503,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1225,"dirs.run_cpu_time_us":262,"dirs.run_wall_time_us":1245,"lbm_read_time_us":527,"lbm_reads_lt_1ms":27,"lbm_write_time_us":399,"lbm_writes_lt_1ms":9,"mutex_wait_us":1759,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":9077,"threads_started":1,"update_count":24}
I20260809 07:58:47.460810 29392 ts_tablet_manager.cc:1929] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.460965 29392 log.cc:1199] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/edd67bd7518a43ec94443bc1ed822a57
I20260809 07:58:47.461097 29394 log.cc:1199] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/edd67bd7518a43ec94443bc1ed822a57
I20260809 07:58:47.461474 29314 consensus_queue.cc:1048] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.461771 29393 tablet_replica.cc:333] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.461913 29392 ts_tablet_manager.cc:1950] T edd67bd7518a43ec94443bc1ed822a57 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.461984 29394 ts_tablet_manager.cc:1950] T edd67bd7518a43ec94443bc1ed822a57 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.462028 29393 raft_consensus.cc:2243] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.462216 29393 raft_consensus.cc:2272] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.462944 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet c0862fc949b040b09e9dd945b1bc3e37 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.463090 29394 tablet_replica.cc:333] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.463222 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet edd67bd7518a43ec94443bc1ed822a57 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.463272 29393 ts_tablet_manager.cc:1916] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.463441 29394 raft_consensus.cc:2243] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.463510 29392 tablet_replica.cc:333] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.463761 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet edd67bd7518a43ec94443bc1ed822a57 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.463786 29392 raft_consensus.cc:2243] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.463820 29394 raft_consensus.cc:2272] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.464201 29392 raft_consensus.cc:2272] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.464870 29198 consensus_queue.cc:1048] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.465392 29392 ts_tablet_manager.cc:1916] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.469798 29393 ts_tablet_manager.cc:1929] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.469924 29393 log.cc:1199] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/e9e62dc7df994bc8ace03052264aa9a2
I20260809 07:58:47.470368 29393 ts_tablet_manager.cc:1950] T e9e62dc7df994bc8ace03052264aa9a2 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.471172 29393 tablet_replica.cc:333] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.471364 29393 raft_consensus.cc:2243] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.471544 29393 raft_consensus.cc:2272] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.472705 29394 ts_tablet_manager.cc:1916] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.474574 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet e9e62dc7df994bc8ace03052264aa9a2 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.476894 29392 ts_tablet_manager.cc:1929] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.478080 29392 log.cc:1199] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/c0862fc949b040b09e9dd945b1bc3e37
I20260809 07:58:47.478614 29392 ts_tablet_manager.cc:1950] T c0862fc949b040b09e9dd945b1bc3e37 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.479538 29392 tablet_replica.cc:333] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.479709 29393 ts_tablet_manager.cc:1916] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.479755 29392 raft_consensus.cc:2243] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.479839 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet c0862fc949b040b09e9dd945b1bc3e37 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.480055 29392 raft_consensus.cc:2272] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.484030 29389 consensus_queue.cc:1048] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.489426 29394 ts_tablet_manager.cc:1929] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.489583 29394 log.cc:1199] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/e9e62dc7df994bc8ace03052264aa9a2
I20260809 07:58:47.490078 29394 ts_tablet_manager.cc:1950] T e9e62dc7df994bc8ace03052264aa9a2 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.491168 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet e9e62dc7df994bc8ace03052264aa9a2 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.492431 29392 ts_tablet_manager.cc:1916] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.494632 29393 ts_tablet_manager.cc:1929] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.494817 29393 log.cc:1199] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/0255a05235b3407d98a10bf04b0c8080
I20260809 07:58:47.495343 29393 ts_tablet_manager.cc:1950] T 0255a05235b3407d98a10bf04b0c8080 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.496347 29393 tablet_replica.cc:333] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.496542 29393 raft_consensus.cc:2243] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.496924 29393 raft_consensus.cc:2272] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.497131 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 0255a05235b3407d98a10bf04b0c8080 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.497239 29394 tablet_replica.cc:333] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.497447 29394 raft_consensus.cc:2243] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.497701 29394 raft_consensus.cc:2272] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.498761 29394 ts_tablet_manager.cc:1916] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.502537 29393 ts_tablet_manager.cc:1916] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.504959 29392 ts_tablet_manager.cc:1929] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.505167 29392 log.cc:1199] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/4e7b2737e49c41438b6478656c6df0a6
I20260809 07:58:47.506623 29392 ts_tablet_manager.cc:1950] T 4e7b2737e49c41438b6478656c6df0a6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.507647 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 4e7b2737e49c41438b6478656c6df0a6 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.507974 29392 tablet_replica.cc:333] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.508209 29392 raft_consensus.cc:2243] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.508555 29392 raft_consensus.cc:2272] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.510984 29392 ts_tablet_manager.cc:1916] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.512154 29393 ts_tablet_manager.cc:1929] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.512286 29393 log.cc:1199] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/4e7b2737e49c41438b6478656c6df0a6
I20260809 07:58:47.512728 29393 ts_tablet_manager.cc:1950] T 4e7b2737e49c41438b6478656c6df0a6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.513833 29393 tablet_replica.cc:333] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.514039 29393 raft_consensus.cc:2243] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.514132 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.514235 29393 raft_consensus.cc:2272] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.514645 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
W20260809 07:58:47.514684 28565 consensus_peers.cc:597] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 -> Peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Couldn't send request to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0255a05235b3407d98a10bf04b0c8080. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:47.518996 29392 ts_tablet_manager.cc:1929] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.519188 29392 log.cc:1199] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/7dce89653b584c46a1128e0c9d43356d
I20260809 07:58:47.519724 29392 ts_tablet_manager.cc:1950] T 7dce89653b584c46a1128e0c9d43356d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.520743 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 7dce89653b584c46a1128e0c9d43356d (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.521100 29392 tablet_replica.cc:333] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.521381 29392 raft_consensus.cc:2243] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.521605 29392 raft_consensus.cc:2272] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.522541 29392 ts_tablet_manager.cc:1916] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.517575 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 4e7b2737e49c41438b6478656c6df0a6 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.528805 29392 ts_tablet_manager.cc:1929] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.528970 29392 log.cc:1199] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/0255a05235b3407d98a10bf04b0c8080
I20260809 07:58:47.529358 29372 raft_consensus.cc:493] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.529532 29372 raft_consensus.cc:515] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.529678 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.013s sys 0.000s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":148,"dirs.run_wall_time_us":968,"drs_written":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1011,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260809 07:58:47.530133 29372 leader_election.cc:290] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:47.530484 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.247496
I20260809 07:58:47.530714 29394 ts_tablet_manager.cc:1929] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.530850 29394 log.cc:1199] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/c0862fc949b040b09e9dd945b1bc3e37
I20260809 07:58:47.531214 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:47.531505 29394 ts_tablet_manager.cc:1950] T c0862fc949b040b09e9dd945b1bc3e37 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.531458 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad6dd9c01dd45f3a16a9be616205018" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:47.531538 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:47.531865 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.247496
I20260809 07:58:47.532212 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad6dd9c01dd45f3a16a9be616205018" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:47.532470 28647 raft_consensus.cc:2393] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a091b20dbc4ed2b3bf6c960df964c5 in current term 1: Already voted for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in this term.
I20260809 07:58:47.532922 29393 ts_tablet_manager.cc:1916] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.533818 28740 leader_election.cc:304] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:47.534279 29372 raft_consensus.cc:3060] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.537086 29372 raft_consensus.cc:2749] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:47.537670 29392 ts_tablet_manager.cc:1950] T 0255a05235b3407d98a10bf04b0c8080 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.537629 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.537922 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.539350 29393 ts_tablet_manager.cc:1929] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.539486 29393 log.cc:1199] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/7dce89653b584c46a1128e0c9d43356d
I20260809 07:58:47.539949 29393 ts_tablet_manager.cc:1950] T 7dce89653b584c46a1128e0c9d43356d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.541077 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 7dce89653b584c46a1128e0c9d43356d (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.541462 29393 tablet_replica.cc:333] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.541663 29393 raft_consensus.cc:2243] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.541859 29393 raft_consensus.cc:2272] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.542418 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:47.542917 29393 ts_tablet_manager.cc:1916] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.543774 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 0255a05235b3407d98a10bf04b0c8080 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.544032 29392 tablet_replica.cc:333] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.544212 29392 raft_consensus.cc:2243] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.544355 29392 raft_consensus.cc:2272] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.544531 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.013s sys 0.000s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":42,"dirs.run_cpu_time_us":156,"dirs.run_wall_time_us":849,"drs_written":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6582,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":17}
I20260809 07:58:47.545969 29392 ts_tablet_manager.cc:1916] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.546173 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.247498
I20260809 07:58:47.546753 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:47.547025 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:47.547338 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4,"spinlock_wait_cycles":6400}
I20260809 07:58:47.547721 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.548055 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.550963 29393 ts_tablet_manager.cc:1929] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.551079 29393 log.cc:1199] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/7bf1f8cb2d2a43668e0aad3ef4052be6
I20260809 07:58:47.551548 29393 ts_tablet_manager.cc:1950] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.552488 29392 ts_tablet_manager.cc:1929] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.552604 29392 log.cc:1199] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/e9e62dc7df994bc8ace03052264aa9a2
I20260809 07:58:47.552618 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.552942 29393 tablet_replica.cc:333] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.553030 29392 ts_tablet_manager.cc:1950] T e9e62dc7df994bc8ace03052264aa9a2 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.553153 29393 raft_consensus.cc:2243] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.553470 29393 pending_rounds.cc:70] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Trying to abort 1 pending ops.
I20260809 07:58:47.553570 29393 pending_rounds.cc:77] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316530491493937152 op_type: NO_OP noop_request { }
I20260809 07:58:47.553686 29393 raft_consensus.cc:2889] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 07:58:47.553807 29393 raft_consensus.cc:2272] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.554059 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet e9e62dc7df994bc8ace03052264aa9a2 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.554497 29392 tablet_replica.cc:333] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.554674 29392 raft_consensus.cc:2243] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.554869 29393 ts_tablet_manager.cc:1916] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.555015 29392 raft_consensus.cc:2272] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.555871 29392 ts_tablet_manager.cc:1916] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.562922 29393 ts_tablet_manager.cc:1929] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.563058 29314 consensus_queue.cc:1048] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:58:47.567430 28566 consensus_peers.cc:597] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0255a05235b3407d98a10bf04b0c8080. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:47.571518 29372 consensus_queue.cc:1048] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.576160 29340 consensus_queue.cc:1048] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.563071 29393 log.cc:1199] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/28a2e99f189940be8e9fb9981c740468
I20260809 07:58:47.577790 29393 ts_tablet_manager.cc:1950] T 28a2e99f189940be8e9fb9981c740468 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.578586 29392 ts_tablet_manager.cc:1929] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.578720 29392 log.cc:1199] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/7bf1f8cb2d2a43668e0aad3ef4052be6
I20260809 07:58:47.578766 29393 tablet_replica.cc:333] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.579005 29393 raft_consensus.cc:2243] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.579227 29392 ts_tablet_manager.cc:1950] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.579380 29393 raft_consensus.cc:2272] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.579567 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 28a2e99f189940be8e9fb9981c740468 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.580225 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.580314 29392 tablet_replica.cc:333] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.580503 29392 raft_consensus.cc:2243] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.580687 29392 raft_consensus.cc:2272] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.581795 29392 ts_tablet_manager.cc:1916] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.582545 29393 ts_tablet_manager.cc:1916] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.587705 29001 consensus_queue.cc:1048] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:47.588829 29392 ts_tablet_manager.cc:1929] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 07:58:47.588992 29392 log.cc:1199] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/28a2e99f189940be8e9fb9981c740468
I20260809 07:58:47.589474 29392 ts_tablet_manager.cc:1950] T 28a2e99f189940be8e9fb9981c740468 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.590406 29392 tablet_replica.cc:333] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.590600 29392 raft_consensus.cc:2243] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.590794 29392 raft_consensus.cc:2272] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.590818 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 28a2e99f189940be8e9fb9981c740468 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.596511 29393 ts_tablet_manager.cc:1929] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.596663 29393 log.cc:1199] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/7592e4276d0d4ddbb0d59cfec840984d
I20260809 07:58:47.597165 29393 ts_tablet_manager.cc:1950] T 7592e4276d0d4ddbb0d59cfec840984d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.598489 29392 ts_tablet_manager.cc:1916] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.600201 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet c0862fc949b040b09e9dd945b1bc3e37 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.603757 29394 tablet_replica.cc:333] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.604035 29394 raft_consensus.cc:2243] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.604537 29394 raft_consensus.cc:2272] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.605003 29393 tablet_replica.cc:333] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.605111 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 7592e4276d0d4ddbb0d59cfec840984d (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.605211 29393 raft_consensus.cc:2243] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.605638 29393 raft_consensus.cc:2272] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.606503 29394 ts_tablet_manager.cc:1916] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.606611 29393 ts_tablet_manager.cc:1916] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.611263 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.611601 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.611918 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.612958 29393 ts_tablet_manager.cc:1929] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.613130 29393 log.cc:1199] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/dafaca6dde5b412e83e9f4cc1edb1f66
I20260809 07:58:47.613621 29393 ts_tablet_manager.cc:1950] T dafaca6dde5b412e83e9f4cc1edb1f66 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.613672 29394 ts_tablet_manager.cc:1929] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.613929 29394 log.cc:1199] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/0255a05235b3407d98a10bf04b0c8080
I20260809 07:58:47.614459 29394 ts_tablet_manager.cc:1950] T 0255a05235b3407d98a10bf04b0c8080 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.614647 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet dafaca6dde5b412e83e9f4cc1edb1f66 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.614727 29393 tablet_replica.cc:333] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.614950 29393 raft_consensus.cc:2243] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.615319 29393 raft_consensus.cc:2272] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.615970 29394 tablet_replica.cc:333] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.616106 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 0255a05235b3407d98a10bf04b0c8080 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.616276 29394 raft_consensus.cc:2243] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.616518 29394 raft_consensus.cc:2272] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.616525 29393 ts_tablet_manager.cc:1916] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.617656 29394 ts_tablet_manager.cc:1916] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.609284 29392 ts_tablet_manager.cc:1929] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.621661 29392 log.cc:1199] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/7592e4276d0d4ddbb0d59cfec840984d
I20260809 07:58:47.622193 29392 ts_tablet_manager.cc:1950] T 7592e4276d0d4ddbb0d59cfec840984d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.622908 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.005s sys 0.000s Metrics: {"bytes_written":649,"delete_count":0,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":13}
I20260809 07:58:47.623050 29393 ts_tablet_manager.cc:1929] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.623198 29392 tablet_replica.cc:333] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.623210 29393 log.cc:1199] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/f152550cc1194a04962055b95dee8e78
I20260809 07:58:47.623432 29392 raft_consensus.cc:2243] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.623450 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 7592e4276d0d4ddbb0d59cfec840984d (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.623638 29393 ts_tablet_manager.cc:1950] T f152550cc1194a04962055b95dee8e78 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.623642 29392 raft_consensus.cc:2272] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.624344 29394 ts_tablet_manager.cc:1929] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.624511 29394 log.cc:1199] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/7dce89653b584c46a1128e0c9d43356d
I20260809 07:58:47.624801 29393 tablet_replica.cc:333] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.625046 29393 raft_consensus.cc:2243] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.625234 29394 ts_tablet_manager.cc:1950] T 7dce89653b584c46a1128e0c9d43356d P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.625470 29393 pending_rounds.cc:70] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Trying to abort 1 pending ops.
I20260809 07:58:47.625586 29393 pending_rounds.cc:77] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316530492361154560 op_type: NO_OP noop_request { }
I20260809 07:58:47.625731 29393 raft_consensus.cc:2889] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 07:58:47.625867 29393 raft_consensus.cc:2272] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.626516 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet f152550cc1194a04962055b95dee8e78 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.626819 29392 ts_tablet_manager.cc:1916] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.626929 29393 ts_tablet_manager.cc:1916] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.630486 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.005s sys 0.000s Metrics: {"bytes_written":459,"delete_count":0,"lbm_write_time_us":179,"lbm_writes_lt_1ms":4,"mutex_wait_us":6962,"reinsert_count":0,"spinlock_wait_cycles":5591936,"update_count":3}
I20260809 07:58:47.631377 29394 tablet_replica.cc:333] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.631677 29394 raft_consensus.cc:2243] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.631719 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":17401}
I20260809 07:58:47.631950 29394 raft_consensus.cc:2272] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.632196 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 7dce89653b584c46a1128e0c9d43356d (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.633106 29394 ts_tablet_manager.cc:1916] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.633441 29393 ts_tablet_manager.cc:1929] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.633584 29393 log.cc:1199] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/ba22ea1e78504b34ba032ce8575f0890
I20260809 07:58:47.634006 29393 ts_tablet_manager.cc:1950] T ba22ea1e78504b34ba032ce8575f0890 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.635004 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet ba22ea1e78504b34ba032ce8575f0890 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.635147 29393 tablet_replica.cc:333] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.635325 29393 raft_consensus.cc:2243] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.635511 29393 raft_consensus.cc:2272] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.635645 29392 ts_tablet_manager.cc:1929] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.635812 29392 log.cc:1199] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/dafaca6dde5b412e83e9f4cc1edb1f66
I20260809 07:58:47.635924 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.104s user 0.022s sys 0.011s Metrics: {"bytes_written":28003,"cfile_cache_hit":15,"cfile_cache_hit_bytes":21212,"cfile_cache_miss":37,"cfile_cache_miss_bytes":30788,"cfile_init":11,"delta_iterators_relevant":10,"dirs.queue_time_us":51,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1082,"drs_written":1,"lbm_read_time_us":8412,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":80,"lbm_write_time_us":3800,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":139420,"rows_written":71}
I20260809 07:58:47.636288 29392 ts_tablet_manager.cc:1950] T dafaca6dde5b412e83e9f4cc1edb1f66 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.636478 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.637104 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3861 bytes on disk
I20260809 07:58:47.637317 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet dafaca6dde5b412e83e9f4cc1edb1f66 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.637456 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3861 bytes on disk
I20260809 07:58:47.637734 29392 tablet_replica.cc:333] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.637884 29393 ts_tablet_manager.cc:1916] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.637795 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":102,"lbm_reads_lt_1ms":4}
I20260809 07:58:47.637913 29392 raft_consensus.cc:2243] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.638085 29392 raft_consensus.cc:2272] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.638082 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.639089 29392 ts_tablet_manager.cc:1916] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.640511 29394 ts_tablet_manager.cc:1929] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.640656 29394 log.cc:1199] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/4e7b2737e49c41438b6478656c6df0a6
I20260809 07:58:47.641098 29394 ts_tablet_manager.cc:1950] T 4e7b2737e49c41438b6478656c6df0a6 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.642340 29394 tablet_replica.cc:333] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.642529 29394 raft_consensus.cc:2243] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.642697 29394 raft_consensus.cc:2272] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.642853 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 4e7b2737e49c41438b6478656c6df0a6 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.643769 29394 ts_tablet_manager.cc:1916] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.644522 28429 tablet_service.cc:1511] Processing CreateTablet for tablet e73d79f178374da8b18a76cb1e350939 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.644950 29393 ts_tablet_manager.cc:1929] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.644958 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73d79f178374da8b18a76cb1e350939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.645150 29393 log.cc:1199] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/744b57379dd54528a734afcedf8128b7
I20260809 07:58:47.645673 29393 ts_tablet_manager.cc:1950] T 744b57379dd54528a734afcedf8128b7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.646130 29392 ts_tablet_manager.cc:1929] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.646237 29392 log.cc:1199] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/f152550cc1194a04962055b95dee8e78
I20260809 07:58:47.646760 29392 ts_tablet_manager.cc:1950] T f152550cc1194a04962055b95dee8e78 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.646773 29393 tablet_replica.cc:333] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.646971 29393 raft_consensus.cc:2243] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.647176 29393 raft_consensus.cc:2272] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.647786 29392 tablet_replica.cc:333] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.647887 29393 ts_tablet_manager.cc:1916] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.647964 29392 raft_consensus.cc:2243] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.648100 29392 raft_consensus.cc:2272] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.648943 29392 ts_tablet_manager.cc:1916] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.649360 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet f152550cc1194a04962055b95dee8e78 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.650290 28797 tablet_service.cc:1511] Processing CreateTablet for tablet e73d79f178374da8b18a76cb1e350939 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.650720 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73d79f178374da8b18a76cb1e350939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.650807 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 43464de37a144d689c41c6f37d20f6f0 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.651196 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43464de37a144d689c41c6f37d20f6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.653661 28794 tablet_service.cc:1511] Processing CreateTablet for tablet ffcb1553e25545ebb774d72295e65fbf (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.654088 28794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffcb1553e25545ebb774d72295e65fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.654217 29393 ts_tablet_manager.cc:1929] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 07:58:47.654331 29393 log.cc:1199] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/0bad7eb37a784618b13b2263c0215b21
I20260809 07:58:47.654676 29393 ts_tablet_manager.cc:1950] T 0bad7eb37a784618b13b2263c0215b21 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.655576 29392 ts_tablet_manager.cc:1929] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 07:58:47.655601 29393 tablet_replica.cc:333] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.655709 29392 log.cc:1199] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/ba22ea1e78504b34ba032ce8575f0890
I20260809 07:58:47.655746 29393 raft_consensus.cc:2243] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.655781 28782 tablet_service.cc:1511] Processing CreateTablet for tablet cda618ed0c4640839f31835d02f906c9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.655957 29393 raft_consensus.cc:2272] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.656100 29392 ts_tablet_manager.cc:1950] T ba22ea1e78504b34ba032ce8575f0890 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.656203 28782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cda618ed0c4640839f31835d02f906c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.656950 29393 ts_tablet_manager.cc:1916] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.657085 29392 tablet_replica.cc:333] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.657276 29392 raft_consensus.cc:2243] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.657303 28795 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee878d7fd574bac94938eb05a9612c9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.657536 29392 raft_consensus.cc:2272] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.657711 28795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee878d7fd574bac94938eb05a9612c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.658308 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet ba22ea1e78504b34ba032ce8575f0890 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.658679 29392 ts_tablet_manager.cc:1916] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.659845 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 744b57379dd54528a734afcedf8128b7 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.660065 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 0bad7eb37a784618b13b2263c0215b21 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.660772 28603 tablet_service.cc:1511] Processing CreateTablet for tablet e73d79f178374da8b18a76cb1e350939 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.660914 28619 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee878d7fd574bac94938eb05a9612c9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.661221 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73d79f178374da8b18a76cb1e350939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.661149 28427 tablet_service.cc:1511] Processing CreateTablet for tablet ffcb1553e25545ebb774d72295e65fbf (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.661615 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffcb1553e25545ebb774d72295e65fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.661654 28602 tablet_service.cc:1511] Processing CreateTablet for tablet cda618ed0c4640839f31835d02f906c9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.662014 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cda618ed0c4640839f31835d02f906c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.665637 28620 tablet_service.cc:1511] Processing CreateTablet for tablet 43464de37a144d689c41c6f37d20f6f0 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.666080 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43464de37a144d689c41c6f37d20f6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.668596 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee878d7fd574bac94938eb05a9612c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.669667 28614 tablet_service.cc:1511] Processing CreateTablet for tablet ffcb1553e25545ebb774d72295e65fbf (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.670081 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffcb1553e25545ebb774d72295e65fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.670081 29263 tablet_bootstrap.cc:492] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:47.670919 28432 tablet_service.cc:1511] Processing CreateTablet for tablet cda618ed0c4640839f31835d02f906c9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.671283 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cda618ed0c4640839f31835d02f906c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.671891 29263 tablet_bootstrap.cc:654] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.673863 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee878d7fd574bac94938eb05a9612c9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.674299 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee878d7fd574bac94938eb05a9612c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.677755 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 43464de37a144d689c41c6f37d20f6f0 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.678200 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43464de37a144d689c41c6f37d20f6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.679509 28437 tablet_service.cc:1511] Processing CreateTablet for tablet 40faafbbf8c646aea67e679524df2cc5 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.679855 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40faafbbf8c646aea67e679524df2cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.680344 28617 tablet_service.cc:1511] Processing CreateTablet for tablet 40faafbbf8c646aea67e679524df2cc5 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.680750 28617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40faafbbf8c646aea67e679524df2cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.687075 28797 tablet_service.cc:1511] Processing CreateTablet for tablet 40faafbbf8c646aea67e679524df2cc5 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.687553 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40faafbbf8c646aea67e679524df2cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.690784 28620 tablet_service.cc:1511] Processing CreateTablet for tablet 56e4487e4b164b49903280601c1b835d (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.691190 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e4487e4b164b49903280601c1b835d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.691434 29405 tablet_bootstrap.cc:492] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:47.692526 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 984b64e2ac0441508af6773002acdb97 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.693704 28603 tablet_service.cc:1511] Processing CreateTablet for tablet 984b64e2ac0441508af6773002acdb97 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.694216 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984b64e2ac0441508af6773002acdb97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.698342 28794 tablet_service.cc:1511] Processing CreateTablet for tablet 984b64e2ac0441508af6773002acdb97 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.699124 28794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984b64e2ac0441508af6773002acdb97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.700006 29405 tablet_bootstrap.cc:654] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.701530 28795 tablet_service.cc:1511] Processing CreateTablet for tablet 56e4487e4b164b49903280601c1b835d (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.701946 28795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e4487e4b164b49903280601c1b835d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.702167 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984b64e2ac0441508af6773002acdb97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.702109 28437 tablet_service.cc:1511] Processing CreateTablet for tablet 56e4487e4b164b49903280601c1b835d (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.702540 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e4487e4b164b49903280601c1b835d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.702628 29393 ts_tablet_manager.cc:1929] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.707343 28617 tablet_service.cc:1511] Processing CreateTablet for tablet 0dbe33c93dd945feb415cf918022f0d9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.707804 28617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dbe33c93dd945feb415cf918022f0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.708209 29393 log.cc:1199] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/1b65040b5e344742890ad101393efc48
I20260809 07:58:47.708276 28432 tablet_service.cc:1511] Processing CreateTablet for tablet 0dbe33c93dd945feb415cf918022f0d9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.708693 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dbe33c93dd945feb415cf918022f0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.708761 29393 ts_tablet_manager.cc:1950] T 1b65040b5e344742890ad101393efc48 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.709626 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 907d3db7702546e58c219fc805be8f7c (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.709884 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 1b65040b5e344742890ad101393efc48 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.710109 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907d3db7702546e58c219fc805be8f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.702425 29392 ts_tablet_manager.cc:1929] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.711019 29392 log.cc:1199] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/744b57379dd54528a734afcedf8128b7
I20260809 07:58:47.711493 29392 ts_tablet_manager.cc:1950] T 744b57379dd54528a734afcedf8128b7 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.711712 29393 tablet_replica.cc:333] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:47.711905 29393 raft_consensus.cc:2243] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.712283 29393 raft_consensus.cc:2272] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.712530 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 744b57379dd54528a734afcedf8128b7 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.713827 29394 ts_tablet_manager.cc:1929] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.713814 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 0dbe33c93dd945feb415cf918022f0d9 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.714020 29394 log.cc:1199] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/7bf1f8cb2d2a43668e0aad3ef4052be6
I20260809 07:58:47.714290 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dbe33c93dd945feb415cf918022f0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.714534 29394 ts_tablet_manager.cc:1950] T 7bf1f8cb2d2a43668e0aad3ef4052be6 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.715049 29393 ts_tablet_manager.cc:1916] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.715471 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 7bf1f8cb2d2a43668e0aad3ef4052be6 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.715591 29392 tablet_replica.cc:333] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.715819 29392 raft_consensus.cc:2243] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.715984 29392 raft_consensus.cc:2272] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.716821 29392 ts_tablet_manager.cc:1916] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.717691 29394 tablet_replica.cc:333] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.717922 29394 raft_consensus.cc:2243] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.718072 29394 raft_consensus.cc:2272] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.718742 29394 ts_tablet_manager.cc:1916] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.722906 29393 ts_tablet_manager.cc:1929] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.723102 29393 log.cc:1199] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/1ad6dd9c01dd45f3a16a9be616205018
I20260809 07:58:47.723675 29393 ts_tablet_manager.cc:1950] T 1ad6dd9c01dd45f3a16a9be616205018 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:47.724818 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 1ad6dd9c01dd45f3a16a9be616205018 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.733752 28620 tablet_service.cc:1511] Processing CreateTablet for tablet 907d3db7702546e58c219fc805be8f7c (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.734238 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907d3db7702546e58c219fc805be8f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.736356 28603 tablet_service.cc:1511] Processing CreateTablet for tablet 01bd743cf4eb439eb0e7b86655b630ec (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.736779 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01bd743cf4eb439eb0e7b86655b630ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.737363 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 5c26d23e8037447c9fdfee1c82f54790 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.737794 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c26d23e8037447c9fdfee1c82f54790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.739101 28787 tablet_service.cc:1511] Processing CreateTablet for tablet 01bd743cf4eb439eb0e7b86655b630ec (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.739497 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01bd743cf4eb439eb0e7b86655b630ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.741577 28602 tablet_service.cc:1511] Processing CreateTablet for tablet 5c26d23e8037447c9fdfee1c82f54790 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.742198 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c26d23e8037447c9fdfee1c82f54790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.742164 28432 tablet_service.cc:1511] Processing CreateTablet for tablet 01bd743cf4eb439eb0e7b86655b630ec (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.742551 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01bd743cf4eb439eb0e7b86655b630ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.743729 28797 tablet_service.cc:1511] Processing CreateTablet for tablet 907d3db7702546e58c219fc805be8f7c (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.745616 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907d3db7702546e58c219fc805be8f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.746114 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 7ffe5276280a42e3830a3b6551950020 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.746549 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffe5276280a42e3830a3b6551950020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.745687 28795 tablet_service.cc:1511] Processing CreateTablet for tablet 5c26d23e8037447c9fdfee1c82f54790 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.747416 28782 tablet_service.cc:1511] Processing CreateTablet for tablet 7ffe5276280a42e3830a3b6551950020 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.749168 28782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffe5276280a42e3830a3b6551950020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.749626 28614 tablet_service.cc:1511] Processing CreateTablet for tablet 7ffe5276280a42e3830a3b6551950020 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.749989 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ffe5276280a42e3830a3b6551950020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.750202 29392 ts_tablet_manager.cc:1929] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 07:58:47.750353 29392 log.cc:1199] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/0bad7eb37a784618b13b2263c0215b21
I20260809 07:58:47.750778 29392 ts_tablet_manager.cc:1950] T 0bad7eb37a784618b13b2263c0215b21 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.751874 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 0bad7eb37a784618b13b2263c0215b21 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.753193 29263 tablet_bootstrap.cc:492] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:47.753387 29263 ts_tablet_manager.cc:1403] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.083s user 0.004s sys 0.004s
I20260809 07:58:47.753410 29394 ts_tablet_manager.cc:1929] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 07:58:47.753549 29394 log.cc:1199] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/28a2e99f189940be8e9fb9981c740468
I20260809 07:58:47.753937 29394 ts_tablet_manager.cc:1950] T 28a2e99f189940be8e9fb9981c740468 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.754343 29263 raft_consensus.cc:359] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.754558 29263 raft_consensus.cc:385] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.754673 29263 raft_consensus.cc:740] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.755014 29392 tablet_replica.cc:333] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.754918 29263 consensus_queue.cc:260] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.755072 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 28a2e99f189940be8e9fb9981c740468 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.755853 29392 raft_consensus.cc:2243] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.756511 29392 raft_consensus.cc:2272] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.757745 29263 ts_tablet_manager.cc:1434] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:47.758096 29263 tablet_bootstrap.cc:492] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:47.758451 29392 ts_tablet_manager.cc:1916] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.761525 29263 tablet_bootstrap.cc:654] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.763191 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.126s user 0.005s sys 0.000s Metrics: {"bytes_written":697,"delete_count":0,"lbm_write_time_us":185,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":16}
I20260809 07:58:47.764374 28795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c26d23e8037447c9fdfee1c82f54790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.765250 29392 ts_tablet_manager.cc:1929] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.766062 29405 tablet_bootstrap.cc:492] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:47.766249 29405 ts_tablet_manager.cc:1403] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.075s user 0.009s sys 0.000s
I20260809 07:58:47.766459 28138 tablet.cc:2106] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260809 07:58:47.767313 29405 raft_consensus.cc:359] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.767555 29405 raft_consensus.cc:385] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.767664 29405 raft_consensus.cc:740] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.767906 29405 consensus_queue.cc:260] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.768219 29394 tablet_replica.cc:333] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.768321 29408 tablet_bootstrap.cc:492] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:47.768407 29394 raft_consensus.cc:2243] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.769225 29394 raft_consensus.cc:2272] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.769415 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 70a22d70306946e3812d58884686f10c (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.769797 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a22d70306946e3812d58884686f10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.770545 29394 ts_tablet_manager.cc:1916] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.774760 28602 tablet_service.cc:1511] Processing CreateTablet for tablet 70a22d70306946e3812d58884686f10c (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.775211 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a22d70306946e3812d58884686f10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.777918 29392 log.cc:1199] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/1ad6dd9c01dd45f3a16a9be616205018
I20260809 07:58:47.778578 29392 ts_tablet_manager.cc:1950] T 1ad6dd9c01dd45f3a16a9be616205018 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.779119 28782 tablet_service.cc:1511] Processing CreateTablet for tablet 70a22d70306946e3812d58884686f10c (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.779551 28782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a22d70306946e3812d58884686f10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.779843 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 1ad6dd9c01dd45f3a16a9be616205018 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.780135 29405 ts_tablet_manager.cc:1434] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260809 07:58:47.780202 29408 tablet_bootstrap.cc:654] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.781220 29405 tablet_bootstrap.cc:492] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:47.783586 29405 tablet_bootstrap.cc:654] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.786448 29394 ts_tablet_manager.cc:1929] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.786818 29392 tablet_replica.cc:333] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:47.787144 29392 raft_consensus.cc:2243] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.787828 29392 raft_consensus.cc:2272] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.788182 29394 log.cc:1199] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/7592e4276d0d4ddbb0d59cfec840984d
I20260809 07:58:47.789011 29394 ts_tablet_manager.cc:1950] T 7592e4276d0d4ddbb0d59cfec840984d P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.790750 29392 ts_tablet_manager.cc:1916] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.790814 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.253s user 0.006s sys 0.019s Metrics: {"dirs.queue_time_us":43466,"dirs.run_cpu_time_us":813,"dirs.run_wall_time_us":4416,"lbm_write_time_us":411,"lbm_writes_lt_1ms":7,"num_input_rowsets":0}
I20260809 07:58:47.792567 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 7592e4276d0d4ddbb0d59cfec840984d (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.794845 29394 tablet_replica.cc:333] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.795002 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 00164db27d9243e08c9b667d92c89297 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.795537 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00164db27d9243e08c9b667d92c89297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.797328 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 160d7600198d40e3a3470352eb70b2c6 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.798256 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160d7600198d40e3a3470352eb70b2c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.798800 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 160d7600198d40e3a3470352eb70b2c6 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.799208 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160d7600198d40e3a3470352eb70b2c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.799608 29394 raft_consensus.cc:2243] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.800048 29394 raft_consensus.cc:2272] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.802435 29394 ts_tablet_manager.cc:1916] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.805759 28794 tablet_service.cc:1511] Processing CreateTablet for tablet 00164db27d9243e08c9b667d92c89297 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.795946 28617 tablet_service.cc:1511] Processing CreateTablet for tablet 00164db27d9243e08c9b667d92c89297 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.798058 28614 tablet_service.cc:1511] Processing CreateTablet for tablet 160d7600198d40e3a3470352eb70b2c6 (DEFAULT_TABLE table=table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:47.806324 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160d7600198d40e3a3470352eb70b2c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.813602 28617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00164db27d9243e08c9b667d92c89297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.814743 29405 tablet_bootstrap.cc:492] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:47.814956 29405 ts_tablet_manager.cc:1403] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.005s
I20260809 07:58:47.816135 29405 raft_consensus.cc:359] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.816602 29405 raft_consensus.cc:385] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.816831 29405 raft_consensus.cc:740] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.817250 29405 consensus_queue.cc:260] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.821135 29405 ts_tablet_manager.cc:1434] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260809 07:58:47.822341 29405 tablet_bootstrap.cc:492] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:47.826944 29405 tablet_bootstrap.cc:654] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.828147 28794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00164db27d9243e08c9b667d92c89297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:47.832049 29408 tablet_bootstrap.cc:492] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:47.832194 29408 ts_tablet_manager.cc:1403] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.000s
I20260809 07:58:47.832289 29394 ts_tablet_manager.cc:1929] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.832413 29394 log.cc:1199] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/dafaca6dde5b412e83e9f4cc1edb1f66
I20260809 07:58:47.832845 29394 ts_tablet_manager.cc:1950] T dafaca6dde5b412e83e9f4cc1edb1f66 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.833029 29408 raft_consensus.cc:359] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.833220 29408 raft_consensus.cc:385] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.833308 29408 raft_consensus.cc:740] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.833946 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet dafaca6dde5b412e83e9f4cc1edb1f66 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.834110 29408 consensus_queue.cc:260] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
W20260809 07:58:47.835284 28566 consensus_peers.cc:597] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 744b57379dd54528a734afcedf8128b7. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:47.835554 29394 tablet_replica.cc:333] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.835726 29394 raft_consensus.cc:2243] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.835922 29394 raft_consensus.cc:2272] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.836931 29394 ts_tablet_manager.cc:1916] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.839066 29408 ts_tablet_manager.cc:1434] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 07:58:47.842265 29405 tablet_bootstrap.cc:492] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:47.842429 29405 ts_tablet_manager.cc:1403] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260809 07:58:47.843145 29392 ts_tablet_manager.cc:1929] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.843302 29392 log.cc:1199] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/1b65040b5e344742890ad101393efc48
I20260809 07:58:47.843319 29408 tablet_bootstrap.cc:492] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:47.843271 29405 raft_consensus.cc:359] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.843472 29405 raft_consensus.cc:385] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.843600 29405 raft_consensus.cc:740] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.843775 29392 ts_tablet_manager.cc:1950] T 1b65040b5e344742890ad101393efc48 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:47.843818 29405 consensus_queue.cc:260] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.844430 29405 ts_tablet_manager.cc:1434] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:47.844904 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 1b65040b5e344742890ad101393efc48 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.845841 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3861 bytes on disk
I20260809 07:58:47.846199 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3861 bytes on disk
I20260809 07:58:47.846477 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 3861 bytes on disk
I20260809 07:58:47.846550 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4,"spinlock_wait_cycles":9472}
I20260809 07:58:47.846827 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.847158 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.848341 29408 tablet_bootstrap.cc:654] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.846572 29405 tablet_bootstrap.cc:492] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:47.850092 29263 tablet_bootstrap.cc:492] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:47.850265 29263 ts_tablet_manager.cc:1403] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.092s user 0.021s sys 0.042s
I20260809 07:58:47.850647 29405 tablet_bootstrap.cc:654] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.851285 29263 raft_consensus.cc:359] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.851500 29263 raft_consensus.cc:385] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.851609 29263 raft_consensus.cc:740] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.852092 29263 consensus_queue.cc:260] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.853669 29263 ts_tablet_manager.cc:1434] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 07:58:47.853986 29263 tablet_bootstrap.cc:492] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:47.857290 29263 tablet_bootstrap.cc:654] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.857863 29394 ts_tablet_manager.cc:1929] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.859295 29394 log.cc:1199] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/f152550cc1194a04962055b95dee8e78
I20260809 07:58:47.859891 29394 ts_tablet_manager.cc:1950] T f152550cc1194a04962055b95dee8e78 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.861092 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet f152550cc1194a04962055b95dee8e78 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.863138 29394 tablet_replica.cc:333] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.863390 29394 raft_consensus.cc:2243] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.863592 29394 raft_consensus.cc:2272] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.864400 29394 ts_tablet_manager.cc:1916] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.871229 29029 raft_consensus.cc:493] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.871459 29029 raft_consensus.cc:515] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.872409 29029 leader_election.cc:290] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.874022 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee878d7fd574bac94938eb05a9612c9" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:47.873950 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.327s user 0.044s sys 0.001s Metrics: {"bytes_written":697,"cfile_cache_hit":15,"cfile_cache_hit_bytes":21212,"cfile_cache_miss":41,"cfile_cache_miss_bytes":31692,"cfile_init":12,"delete_count":0,"delta_iterators_relevant":12,"dirs.queue_time_us":121622,"dirs.run_cpu_time_us":1458,"dirs.run_wall_time_us":7389,"drs_written":1,"lbm_read_time_us":58522,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":88,"lbm_write_time_us":34668,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":31,"mutex_wait_us":26,"num_input_rowsets":2,"peak_mem_usage":139425,"reinsert_count":0,"rows_written":71,"spinlock_wait_cycles":19328,"thread_start_us":15307,"threads_started":3,"update_count":16}
I20260809 07:58:47.873245 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee878d7fd574bac94938eb05a9612c9" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:47.874768 28820 raft_consensus.cc:2468] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:47.875401 28566 leader_election.cc:304] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
W20260809 07:58:47.875698 28565 leader_election.cc:343] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:47.876144 29029 raft_consensus.cc:2804] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:47.876255 29029 raft_consensus.cc:493] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:47.876381 29029 raft_consensus.cc:3060] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.878082 29394 ts_tablet_manager.cc:1929] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260809 07:58:47.878274 29394 log.cc:1199] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/ba22ea1e78504b34ba032ce8575f0890
I20260809 07:58:47.878643 29394 ts_tablet_manager.cc:1950] T ba22ea1e78504b34ba032ce8575f0890 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.879545 29394 tablet_replica.cc:333] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.879745 29394 raft_consensus.cc:2243] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.879738 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet ba22ea1e78504b34ba032ce8575f0890 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.880069 29394 raft_consensus.cc:2272] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.880904 29405 tablet_bootstrap.cc:492] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:47.880997 29394 ts_tablet_manager.cc:1916] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.881469 29405 ts_tablet_manager.cc:1403] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.007s
I20260809 07:58:47.881814 29029 raft_consensus.cc:515] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.882900 29405 raft_consensus.cc:359] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.884052 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee878d7fd574bac94938eb05a9612c9" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
W20260809 07:58:47.884733 28565 leader_election.cc:343] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:47.885689 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee878d7fd574bac94938eb05a9612c9" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:47.885895 28820 raft_consensus.cc:3060] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.887559 29394 ts_tablet_manager.cc:1929] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.887663 29394 log.cc:1199] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/744b57379dd54528a734afcedf8128b7
I20260809 07:58:47.888088 29394 ts_tablet_manager.cc:1950] T 744b57379dd54528a734afcedf8128b7 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.888919 29394 tablet_replica.cc:333] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.889094 29394 raft_consensus.cc:2243] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:47.889308 29394 pending_rounds.cc:70] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Trying to abort 1 pending ops.
I20260809 07:58:47.889398 29394 pending_rounds.cc:77] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316530492780371968 op_type: NO_OP noop_request { }
I20260809 07:58:47.889495 29394 raft_consensus.cc:2889] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260809 07:58:47.889614 29394 raft_consensus.cc:2272] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.889844 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 744b57379dd54528a734afcedf8128b7 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.890367 28820 raft_consensus.cc:2468] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:47.890420 29408 tablet_bootstrap.cc:492] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:47.890517 29263 tablet_bootstrap.cc:492] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:47.890591 29408 ts_tablet_manager.cc:1403] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.047s user 0.013s sys 0.008s
I20260809 07:58:47.890666 29263 ts_tablet_manager.cc:1403] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.007s
I20260809 07:58:47.891019 28566 leader_election.cc:304] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:47.891319 28961 raft_consensus.cc:2804] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:47.891444 28961 raft_consensus.cc:697] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:47.891417 29408 raft_consensus.cc:359] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.891603 29408 raft_consensus.cc:385] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.891664 29408 raft_consensus.cc:740] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.891582 29263 raft_consensus.cc:359] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:47.891743 29263 raft_consensus.cc:385] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.891824 29263 raft_consensus.cc:740] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.891739 28961 consensus_queue.cc:237] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.891880 29408 consensus_queue.cc:260] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.892047 29263 consensus_queue.cc:260] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:47.892303 29408 ts_tablet_manager.cc:1434] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 07:58:47.892539 29263 ts_tablet_manager.cc:1434] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260809 07:58:47.892633 29408 tablet_bootstrap.cc:492] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:47.892853 29263 tablet_bootstrap.cc:492] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:47.896070 29263 tablet_bootstrap.cc:654] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.896416 29408 tablet_bootstrap.cc:654] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.898943 29394 ts_tablet_manager.cc:1916] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.899323 29029 leader_election.cc:290] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.899614 29405 raft_consensus.cc:385] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.900151 29405 raft_consensus.cc:740] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.900831 29405 consensus_queue.cc:260] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.901974 29405 ts_tablet_manager.cc:1434] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20260809 07:58:47.903410 29405 tablet_bootstrap.cc:492] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:47.906121 29405 tablet_bootstrap.cc:654] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.906484 29408 tablet_bootstrap.cc:492] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:47.906657 29408 ts_tablet_manager.cc:1403] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.001s
I20260809 07:58:47.907315 29408 raft_consensus.cc:359] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.907542 29408 raft_consensus.cc:385] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.907646 29408 raft_consensus.cc:740] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.907871 29408 consensus_queue.cc:260] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.908341 29408 ts_tablet_manager.cc:1434] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:47.908622 29408 tablet_bootstrap.cc:492] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:47.910323 29408 tablet_bootstrap.cc:654] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.912956 29394 ts_tablet_manager.cc:1929] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.913133 29394 log.cc:1199] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/0bad7eb37a784618b13b2263c0215b21
I20260809 07:58:47.913767 29394 ts_tablet_manager.cc:1950] T 0bad7eb37a784618b13b2263c0215b21 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.914808 29394 tablet_replica.cc:333] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.915050 29394 raft_consensus.cc:2243] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.915266 29394 raft_consensus.cc:2272] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.916392 29394 ts_tablet_manager.cc:1916] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.916977 28153 catalog_manager.cc:5719] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:47.921811 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 0bad7eb37a784618b13b2263c0215b21 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.925443 29394 ts_tablet_manager.cc:1929] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.925616 29394 log.cc:1199] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/1b65040b5e344742890ad101393efc48
I20260809 07:58:47.926146 29394 ts_tablet_manager.cc:1950] T 1b65040b5e344742890ad101393efc48 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.927294 29394 tablet_replica.cc:333] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:47.927515 29394 raft_consensus.cc:2243] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:47.927740 29394 raft_consensus.cc:2272] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:47.929301 29394 ts_tablet_manager.cc:1916] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:47.932361 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 1b65040b5e344742890ad101393efc48 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.935084 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.935555 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.936055 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.936545 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.936969 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.937294 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.937665 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.937970 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:47.939349 29029 raft_consensus.cc:493] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.939594 29029 raft_consensus.cc:515] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.939761 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.940039 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.940267 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.940765 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.940794 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cda618ed0c4640839f31835d02f906c9" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:47.941035 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:47.941336 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
W20260809 07:58:47.941409 28565 leader_election.cc:343] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:47.941674 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.941915 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:47.942358 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cda618ed0c4640839f31835d02f906c9" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:47.942934 28566 leader_election.cc:343] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:47.943050 28566 leader_election.cc:304] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:47.943316 29389 raft_consensus.cc:2749] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:47.943578 29029 leader_election.cc:290] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.945787 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":212,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.951017 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.006s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":188,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:47.951308 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.005s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":931,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":46,"lbm_read_time_us":107,"lbm_reads_lt_1ms":8,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"mutex_wait_us":4142,"reinsert_count":0,"update_count":17}
I20260809 07:58:47.951947 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.005s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":931,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":689,"lbm_read_time_us":252,"lbm_reads_lt_1ms":8,"lbm_write_time_us":221,"lbm_writes_lt_1ms":4,"mutex_wait_us":5157,"reinsert_count":0,"update_count":17}
I20260809 07:58:47.953012 29405 tablet_bootstrap.cc:492] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:47.953193 29405 ts_tablet_manager.cc:1403] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.000s
I20260809 07:58:47.954151 29405 raft_consensus.cc:359] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:47.954337 29405 raft_consensus.cc:385] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.954416 29405 raft_consensus.cc:740] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.954689 29405 consensus_queue.cc:260] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:47.957355 29408 tablet_bootstrap.cc:492] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:47.957545 29408 ts_tablet_manager.cc:1403] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.049s user 0.015s sys 0.015s
I20260809 07:58:47.958045 29405 ts_tablet_manager.cc:1434] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:58:47.958359 29408 raft_consensus.cc:359] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.959700 29408 raft_consensus.cc:385] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.959803 29408 raft_consensus.cc:740] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.960093 29408 consensus_queue.cc:260] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.960690 29408 ts_tablet_manager.cc:1434] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260809 07:58:47.961609 29405 tablet_bootstrap.cc:492] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:47.962332 29263 tablet_bootstrap.cc:492] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:47.962517 29263 ts_tablet_manager.cc:1403] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.033s
I20260809 07:58:47.963244 29408 tablet_bootstrap.cc:492] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:47.963338 29263 raft_consensus.cc:359] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.963557 29263 raft_consensus.cc:385] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.963647 29263 raft_consensus.cc:740] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.963994 29263 consensus_queue.cc:260] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.964514 29263 ts_tablet_manager.cc:1434] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:47.965615 29405 tablet_bootstrap.cc:654] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.966228 29263 tablet_bootstrap.cc:492] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:47.967528 29408 tablet_bootstrap.cc:654] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.967958 29263 tablet_bootstrap.cc:654] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.970285 29340 raft_consensus.cc:493] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.970425 29340 raft_consensus.cc:515] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.971239 29340 leader_election.cc:290] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.972380 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43464de37a144d689c41c6f37d20f6f0" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:47.973505 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43464de37a144d689c41c6f37d20f6f0" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:47.973701 28648 raft_consensus.cc:2468] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:47.974382 28380 leader_election.cc:304] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
W20260809 07:58:47.974869 28382 leader_election.cc:343] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:47.976087 29340 raft_consensus.cc:2804] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:47.976217 29340 raft_consensus.cc:493] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:47.976315 29340 raft_consensus.cc:3060] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.979787 29340 raft_consensus.cc:515] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.980262 29014 raft_consensus.cc:493] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:47.980526 29014 raft_consensus.cc:515] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.981189 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43464de37a144d689c41c6f37d20f6f0" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:47.981472 29014 leader_election.cc:290] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.981846 28648 raft_consensus.cc:3060] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.984598 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee878d7fd574bac94938eb05a9612c9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:47.985337 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee878d7fd574bac94938eb05a9612c9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:47.985664 28820 raft_consensus.cc:2393] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in current term 1: Already voted for candidate 24137fd377124538a31e92f3b6df0673 in this term.
I20260809 07:58:47.986044 29263 tablet_bootstrap.cc:492] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:47.986214 29263 ts_tablet_manager.cc:1403] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.004s
I20260809 07:58:47.987073 29263 raft_consensus.cc:359] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.987293 29263 raft_consensus.cc:385] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:47.987402 29263 raft_consensus.cc:740] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:47.987633 29263 consensus_queue.cc:260] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.988021 28648 raft_consensus.cc:2468] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:47.990223 29263 ts_tablet_manager.cc:1434] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 07:58:47.991004 29263 tablet_bootstrap.cc:492] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:47.993637 29394 ts_tablet_manager.cc:1929] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:47.993764 29263 tablet_bootstrap.cc:654] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:47.993788 29394 log.cc:1199] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/1ad6dd9c01dd45f3a16a9be616205018
I20260809 07:58:47.994238 29394 ts_tablet_manager.cc:1950] T 1ad6dd9c01dd45f3a16a9be616205018 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:47.995249 28380 leader_election.cc:304] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:47.995405 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 1ad6dd9c01dd45f3a16a9be616205018 (table table-2 [id=fc96e4603f5148449b4d5a5358162240]) successfully deleted
I20260809 07:58:47.995951 28380 leader_election.cc:304] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:47.996114 29340 leader_election.cc:290] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:47.996251 29314 raft_consensus.cc:2804] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:47.996358 29014 raft_consensus.cc:3060] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:47.996670 29314 raft_consensus.cc:697] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:47.997036 29314 consensus_queue.cc:237] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:47.997421 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43464de37a144d689c41c6f37d20f6f0" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:47.997640 28820 raft_consensus.cc:3060] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.001637 28820 raft_consensus.cc:2468] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.003512 29014 raft_consensus.cc:2749] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:48.004333 29408 tablet_bootstrap.cc:492] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.004632 29408 ts_tablet_manager.cc:1403] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.006s
I20260809 07:58:48.005589 29408 raft_consensus.cc:359] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:48.005772 29408 raft_consensus.cc:385] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.005869 29408 raft_consensus.cc:740] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.006142 29408 consensus_queue.cc:260] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:48.006644 29408 ts_tablet_manager.cc:1434] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:48.006968 29408 tablet_bootstrap.cc:492] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.010973 29408 tablet_bootstrap.cc:654] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.015579 28153 catalog_manager.cc:5719] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.018108 29263 tablet_bootstrap.cc:492] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.018296 29263 ts_tablet_manager.cc:1403] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.003s
I20260809 07:58:48.019179 29263 raft_consensus.cc:359] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.019423 29263 raft_consensus.cc:385] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.019531 29263 raft_consensus.cc:740] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.019786 29263 consensus_queue.cc:260] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.020380 29263 ts_tablet_manager.cc:1434] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.023664 29263 tablet_bootstrap.cc:492] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.026453 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.026854 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.027165 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.027530 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.027791 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.028142 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.028271 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.028604 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.028841 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.028966 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.029110 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.029381 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.030836 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.031122 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.031327 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.031800 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.032138 29405 tablet_bootstrap.cc:492] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.032130 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.032343 29405 ts_tablet_manager.cc:1403] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.071s user 0.015s sys 0.006s
I20260809 07:58:48.032430 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.032793 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.033075 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.033416 29405 raft_consensus.cc:359] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.033675 29405 raft_consensus.cc:385] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.033756 29405 raft_consensus.cc:740] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.034107 29405 consensus_queue.cc:260] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.036715 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.037899 29405 ts_tablet_manager.cc:1434] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:58:48.038755 29405 tablet_bootstrap.cc:492] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.040802 29405 tablet_bootstrap.cc:654] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.041127 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1015,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":49,"lbm_read_time_us":194,"lbm_reads_lt_1ms":8,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"mutex_wait_us":3728,"reinsert_count":0,"update_count":18}
I20260809 07:58:48.045101 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.004s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1015,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":82,"lbm_read_time_us":187,"lbm_reads_lt_1ms":8,"lbm_write_time_us":182,"lbm_writes_lt_1ms":4,"mutex_wait_us":4100,"reinsert_count":0,"update_count":18}
I20260809 07:58:48.045655 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":156,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.047888 29263 tablet_bootstrap.cc:654] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.054425 29408 tablet_bootstrap.cc:492] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.054637 29408 ts_tablet_manager.cc:1403] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.007s
I20260809 07:58:48.055545 29408 raft_consensus.cc:359] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.055785 29408 raft_consensus.cc:385] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.055898 29408 raft_consensus.cc:740] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.056186 29408 consensus_queue.cc:260] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.056749 29408 ts_tablet_manager.cc:1434] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:48.058449 29408 tablet_bootstrap.cc:492] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.064152 29408 tablet_bootstrap.cc:654] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.066308 29001 raft_consensus.cc:493] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.066474 29001 raft_consensus.cc:515] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.067225 29001 leader_election.cc:290] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.067878 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40faafbbf8c646aea67e679524df2cc5" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.068142 28461 raft_consensus.cc:2468] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.068145 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40faafbbf8c646aea67e679524df2cc5" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.068468 28648 raft_consensus.cc:2468] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.068701 28740 leader_election.cc:304] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.069037 29001 raft_consensus.cc:2804] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.069159 29001 raft_consensus.cc:493] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.069252 29001 raft_consensus.cc:3060] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.072427 29001 raft_consensus.cc:515] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.073093 29001 leader_election.cc:290] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.073630 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40faafbbf8c646aea67e679524df2cc5" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.073863 28648 raft_consensus.cc:3060] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.076972 28648 raft_consensus.cc:2468] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.077699 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40faafbbf8c646aea67e679524df2cc5" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:48.077932 28461 raft_consensus.cc:3060] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.081221 28461 raft_consensus.cc:2468] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.081539 28740 leader_election.cc:304] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.081903 29001 raft_consensus.cc:2804] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.082085 29001 raft_consensus.cc:697] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:48.082407 29001 consensus_queue.cc:237] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.089720 29405 tablet_bootstrap.cc:492] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.089887 29405 ts_tablet_manager.cc:1403] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.007s
I20260809 07:58:48.090875 29405 raft_consensus.cc:359] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.091107 29405 raft_consensus.cc:385] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.091228 29405 raft_consensus.cc:740] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.091502 29405 consensus_queue.cc:260] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.092140 29405 ts_tablet_manager.cc:1434] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:48.099131 29408 tablet_bootstrap.cc:492] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.099306 29408 ts_tablet_manager.cc:1403] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.003s
I20260809 07:58:48.100158 29408 raft_consensus.cc:359] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.100342 29408 raft_consensus.cc:385] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.100433 29408 raft_consensus.cc:740] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.100649 29408 consensus_queue.cc:260] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.101230 29408 ts_tablet_manager.cc:1434] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.101596 29408 tablet_bootstrap.cc:492] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.101625 29405 tablet_bootstrap.cc:492] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.103559 29408 tablet_bootstrap.cc:654] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.106196 29405 tablet_bootstrap.cc:654] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.108587 28153 catalog_manager.cc:5719] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:48.111172 28961 raft_consensus.cc:493] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.111351 28961 raft_consensus.cc:515] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.112051 28961 leader_election.cc:290] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.112721 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffcb1553e25545ebb774d72295e65fbf" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.112953 28820 raft_consensus.cc:2468] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:48.113490 28566 leader_election.cc:304] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.116640 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffcb1553e25545ebb774d72295e65fbf" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.116889 28461 raft_consensus.cc:2468] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:48.117305 29263 tablet_bootstrap.cc:492] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.117484 29263 ts_tablet_manager.cc:1403] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.094s user 0.012s sys 0.019s
I20260809 07:58:48.118410 29263 raft_consensus.cc:359] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.118629 29263 raft_consensus.cc:385] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.118781 29263 raft_consensus.cc:740] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.119048 29263 consensus_queue.cc:260] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.119570 29263 ts_tablet_manager.cc:1434] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:48.121507 28961 raft_consensus.cc:2804] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.121622 28961 raft_consensus.cc:493] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.121752 28961 raft_consensus.cc:3060] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.125088 29263 tablet_bootstrap.cc:492] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.126503 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.126823 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.127157 29314 raft_consensus.cc:493] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.127265 29263 tablet_bootstrap.cc:654] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.127286 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.127313 29314 raft_consensus.cc:515] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.127599 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.128043 29314 leader_election.cc:290] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.129055 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.129365 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.129652 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.129935 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.129885 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.130190 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.130251 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984b64e2ac0441508af6773002acdb97" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.130481 28648 raft_consensus.cc:2468] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.130460 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.132509 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.132829 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.133073 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.133800 28380 leader_election.cc:304] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.134727 29314 raft_consensus.cc:2804] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.134855 29314 raft_consensus.cc:493] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.134953 29314 raft_consensus.cc:3060] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.134851 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984b64e2ac0441508af6773002acdb97" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.136101 29014 raft_consensus.cc:493] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.136277 29014 raft_consensus.cc:515] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.136402 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.136727 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.137013 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.137094 29014 leader_election.cc:290] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.137362 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.137637 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.138073 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cda618ed0c4640839f31835d02f906c9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.138362 28820 raft_consensus.cc:2468] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.138418 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cda618ed0c4640839f31835d02f906c9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.138631 28648 raft_consensus.cc:2468] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
W20260809 07:58:48.139338 28382 leader_election.cc:343] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:48.139366 28961 raft_consensus.cc:515] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.139715 28382 leader_election.cc:304] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.140328 29014 raft_consensus.cc:2804] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.140465 29014 raft_consensus.cc:493] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.140568 29014 raft_consensus.cc:3060] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.140674 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffcb1553e25545ebb774d72295e65fbf" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:48.140869 28461 raft_consensus.cc:3060] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.141743 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.144486 28461 raft_consensus.cc:2468] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:48.144682 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.002s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":213,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.145085 28565 leader_election.cc:304] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.145468 29029 raft_consensus.cc:2804] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.145618 29029 raft_consensus.cc:697] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:48.145947 29029 consensus_queue.cc:237] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.147293 29014 raft_consensus.cc:515] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.148644 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cda618ed0c4640839f31835d02f906c9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.148857 28648 raft_consensus.cc:3060] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.149906 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffcb1553e25545ebb774d72295e65fbf" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:48.150159 28820 raft_consensus.cc:3060] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.152495 28648 raft_consensus.cc:2468] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.153045 28380 leader_election.cc:304] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.153427 29340 raft_consensus.cc:2804] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.154989 29314 raft_consensus.cc:515] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.155606 28153 catalog_manager.cc:5719] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.156529 28961 leader_election.cc:290] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.157235 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984b64e2ac0441508af6773002acdb97" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.157444 28647 raft_consensus.cc:3060] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.158210 29263 tablet_bootstrap.cc:492] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.158394 29263 ts_tablet_manager.cc:1403] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.006s
I20260809 07:58:48.159260 29263 raft_consensus.cc:359] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.159468 29263 raft_consensus.cc:385] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.159549 29263 raft_consensus.cc:740] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.159796 28647 raft_consensus.cc:2468] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.159809 29263 consensus_queue.cc:260] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.152773 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1036,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":54,"lbm_read_time_us":186,"lbm_reads_lt_1ms":8,"lbm_write_time_us":204,"lbm_writes_lt_1ms":4,"mutex_wait_us":2602,"reinsert_count":0,"update_count":19}
I20260809 07:58:48.160300 28380 leader_election.cc:304] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.160388 29263 ts_tablet_manager.cc:1434] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.161811 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984b64e2ac0441508af6773002acdb97" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:48.162567 29263 tablet_bootstrap.cc:492] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
W20260809 07:58:48.162880 28382 leader_election.cc:343] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:48.163010 28826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cda618ed0c4640839f31835d02f906c9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:48.163102 29314 leader_election.cc:290] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.163198 28826 raft_consensus.cc:3060] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.163563 29014 leader_election.cc:290] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.163790 29014 raft_consensus.cc:2804] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.163928 29014 raft_consensus.cc:697] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:48.164284 29014 consensus_queue.cc:237] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.164767 29263 tablet_bootstrap.cc:654] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.171588 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.171947 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.172271 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.172516 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.172801 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.173066 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.173125 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.173398 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.173552 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.173674 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.173914 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.173990 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.174467 28826 raft_consensus.cc:2468] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.187171 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":13330}
I20260809 07:58:48.187827 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":203,"lbm_writes_lt_1ms":4,"mutex_wait_us":8338,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.189292 28820 raft_consensus.cc:2468] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:48.189600 29314 raft_consensus.cc:493] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.189849 29314 raft_consensus.cc:515] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:48.190311 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.062s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":185,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.190897 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.054s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1036,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1536,"dirs.run_cpu_time_us":143,"dirs.run_wall_time_us":816,"lbm_read_time_us":164,"lbm_reads_lt_1ms":8,"lbm_write_time_us":171,"lbm_writes_lt_1ms":4,"mutex_wait_us":19823,"reinsert_count":0,"update_count":19}
I20260809 07:58:48.191300 29314 leader_election.cc:290] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.191605 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73d79f178374da8b18a76cb1e350939" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.191913 28820 raft_consensus.cc:2468] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.192509 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.192649 28382 leader_election.cc:304] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.192891 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.193019 29314 raft_consensus.cc:2804] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.193154 29314 raft_consensus.cc:493] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.193235 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.193275 29314 raft_consensus.cc:3060] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.193596 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.191869 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.005s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1063,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":91,"lbm_read_time_us":244,"lbm_reads_lt_1ms":8,"lbm_write_time_us":191,"lbm_writes_lt_1ms":4,"mutex_wait_us":6688,"reinsert_count":0,"update_count":20}
I20260809 07:58:48.199398 29314 raft_consensus.cc:515] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:48.202823 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.203943 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.203986 29001 raft_consensus.cc:493] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.204186 29001 raft_consensus.cc:515] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.204564 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.205092 29001 leader_election.cc:290] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.205165 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73d79f178374da8b18a76cb1e350939" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:48.205757 28820 raft_consensus.cc:3060] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.206209 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe33c93dd945feb415cf918022f0d9" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.206230 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe33c93dd945feb415cf918022f0d9" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.206610 29263 tablet_bootstrap.cc:492] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.206795 29263 ts_tablet_manager.cc:1403] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.004s
W20260809 07:58:48.207087 28740 leader_election.cc:343] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:48.207438 28740 leader_election.cc:343] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:48.207646 28740 leader_election.cc:304] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:48.207949 29263 raft_consensus.cc:359] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.208168 29263 raft_consensus.cc:385] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.208283 29263 raft_consensus.cc:740] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.208583 29263 consensus_queue.cc:260] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.208941 29211 raft_consensus.cc:2749] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:48.209263 29263 ts_tablet_manager.cc:1434] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.209297 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.209645 29263 tablet_bootstrap.cc:492] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.212513 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.004s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1063,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":9491,"lbm_read_time_us":222,"lbm_reads_lt_1ms":8,"lbm_write_time_us":192,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":20}
I20260809 07:58:48.213565 29340 raft_consensus.cc:697] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:48.213923 29263 tablet_bootstrap.cc:654] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.215333 28820 raft_consensus.cc:2468] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.215768 29340 consensus_queue.cc:237] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.217437 28382 leader_election.cc:304] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.218763 29451 raft_consensus.cc:2804] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.218961 29451 raft_consensus.cc:697] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:48.219305 29451 consensus_queue.cc:237] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } }
I20260809 07:58:48.224648 29408 tablet_bootstrap.cc:492] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.224905 29408 ts_tablet_manager.cc:1403] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.123s user 0.009s sys 0.001s
I20260809 07:58:48.226061 29408 raft_consensus.cc:359] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.226306 29314 leader_election.cc:290] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.226533 29408 raft_consensus.cc:385] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.226642 29408 raft_consensus.cc:740] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.227092 29408 consensus_queue.cc:260] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
W20260809 07:58:48.229442 28380 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.37.2:38379, user_credentials={real_user=slave}} blocked reactor thread for 61209.7us
I20260809 07:58:48.230331 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73d79f178374da8b18a76cb1e350939" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.230574 28648 raft_consensus.cc:2468] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.231480 29408 ts_tablet_manager.cc:1434] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260809 07:58:48.231858 29408 tablet_bootstrap.cc:492] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.234378 29408 tablet_bootstrap.cc:654] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.236153 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73d79f178374da8b18a76cb1e350939" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.236390 28648 raft_consensus.cc:3060] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.240334 28648 raft_consensus.cc:2468] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.247197 28151 catalog_manager.cc:5719] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.247876 28151 catalog_manager.cc:5719] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.261695 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.261976 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.262197 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.262427 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.262203 28151 catalog_manager.cc:5719] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.264180 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.264467 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.264719 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.264911 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.265079 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121965
I20260809 07:58:48.265276 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.265573 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121965
I20260809 07:58:48.265825 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.266040 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.266270 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.266992 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.267539 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121965
I20260809 07:58:48.267827 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121965
I20260809 07:58:48.268112 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.121965
I20260809 07:58:48.268458 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.268697 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.270836 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.271246 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.274196 29263 tablet_bootstrap.cc:492] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.274403 29263 ts_tablet_manager.cc:1403] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.065s user 0.005s sys 0.003s
I20260809 07:58:48.274439 29408 tablet_bootstrap.cc:492] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.274576 29408 ts_tablet_manager.cc:1403] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.001s
I20260809 07:58:48.275279 29263 raft_consensus.cc:359] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.275380 29408 raft_consensus.cc:359] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.275478 29263 raft_consensus.cc:385] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.275548 29408 raft_consensus.cc:385] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.275552 29263 raft_consensus.cc:740] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.275636 29408 raft_consensus.cc:740] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.275797 29263 consensus_queue.cc:260] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.275833 29408 consensus_queue.cc:260] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.276295 29408 ts_tablet_manager.cc:1434] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.276335 29263 ts_tablet_manager.cc:1434] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.276597 29408 tablet_bootstrap.cc:492] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.276674 29263 tablet_bootstrap.cc:492] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.278981 29408 tablet_bootstrap.cc:654] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.282755 29263 tablet_bootstrap.cc:654] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.288071 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.288345 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.289741 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.289636 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.005s sys 0.000s Metrics: {"bytes_written":454,"delete_count":0,"lbm_write_time_us":158,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 07:58:48.290066 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.290091 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.290313 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.001s sys 0.004s Metrics: {"bytes_written":454,"delete_count":0,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260809 07:58:48.297667 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.004s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":2099,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":3,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.297932 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":7695}
I20260809 07:58:48.299062 29408 tablet_bootstrap.cc:492] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.299259 29408 ts_tablet_manager.cc:1403] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.004s
I20260809 07:58:48.299762 29263 tablet_bootstrap.cc:492] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.299940 29263 ts_tablet_manager.cc:1403] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.007s
I20260809 07:58:48.300038 29408 raft_consensus.cc:359] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.300257 29408 raft_consensus.cc:385] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.300345 29408 raft_consensus.cc:740] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.300594 29408 consensus_queue.cc:260] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.300786 29263 raft_consensus.cc:359] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.300995 29263 raft_consensus.cc:385] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.301100 29263 raft_consensus.cc:740] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.301200 29408 ts_tablet_manager.cc:1434] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.301595 29408 tablet_bootstrap.cc:492] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.301378 29263 consensus_queue.cc:260] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.302032 29405 tablet_bootstrap.cc:492] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.302213 29405 ts_tablet_manager.cc:1403] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.201s user 0.001s sys 0.009s
I20260809 07:58:48.302389 29263 ts_tablet_manager.cc:1434] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 07:58:48.302663 29263 tablet_bootstrap.cc:492] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.302994 29405 raft_consensus.cc:359] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.303218 29405 raft_consensus.cc:385] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.303282 29408 tablet_bootstrap.cc:654] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.303287 29405 raft_consensus.cc:740] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.303565 29405 consensus_queue.cc:260] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.304105 29405 ts_tablet_manager.cc:1434] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.304378 29263 tablet_bootstrap.cc:654] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.304436 29405 tablet_bootstrap.cc:492] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.311509 29001 raft_consensus.cc:493] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.311668 29001 raft_consensus.cc:515] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.312346 29408 tablet_bootstrap.cc:492] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.312352 29001 leader_election.cc:290] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.312479 29408 ts_tablet_manager.cc:1403] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260809 07:58:48.313328 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e4487e4b164b49903280601c1b835d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.313390 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e4487e4b164b49903280601c1b835d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.313550 28461 raft_consensus.cc:2468] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.313562 28648 raft_consensus.cc:2468] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.314002 28740 leader_election.cc:304] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.314275 29001 raft_consensus.cc:2804] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.314396 29001 raft_consensus.cc:493] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.314498 29001 raft_consensus.cc:3060] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.314502 29408 raft_consensus.cc:359] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.314667 29408 raft_consensus.cc:385] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.314761 29408 raft_consensus.cc:740] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.315038 29408 consensus_queue.cc:260] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.315533 29405 tablet_bootstrap.cc:654] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.316092 29408 ts_tablet_manager.cc:1434] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 07:58:48.316831 29408 tablet_bootstrap.cc:492] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.318701 29408 tablet_bootstrap.cc:654] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.320236 29001 raft_consensus.cc:515] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.321432 29263 tablet_bootstrap.cc:492] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.321610 29263 ts_tablet_manager.cc:1403] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.013s
I20260809 07:58:48.322439 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e4487e4b164b49903280601c1b835d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:48.322642 28461 raft_consensus.cc:3060] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.322608 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e4487e4b164b49903280601c1b835d" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.322793 28648 raft_consensus.cc:3060] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.325708 28648 raft_consensus.cc:2468] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.325999 28461 raft_consensus.cc:2468] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.326254 28740 leader_election.cc:304] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.326552 29211 raft_consensus.cc:2804] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.326809 29001 leader_election.cc:290] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.327708 29263 raft_consensus.cc:359] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.327920 29263 raft_consensus.cc:385] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.328006 29263 raft_consensus.cc:740] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.328440 29263 consensus_queue.cc:260] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.329093 29263 ts_tablet_manager.cc:1434] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 07:58:48.329434 29263 tablet_bootstrap.cc:492] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.329874 28961 consensus_queue.cc:1048] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.331108 29263 tablet_bootstrap.cc:654] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.333559 29405 tablet_bootstrap.cc:492] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.333709 29405 ts_tablet_manager.cc:1403] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.003s
I20260809 07:58:48.334626 29405 raft_consensus.cc:359] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.334829 29405 raft_consensus.cc:385] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.334924 29405 raft_consensus.cc:740] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.335187 29405 consensus_queue.cc:260] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.335747 29405 ts_tablet_manager.cc:1434] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:48.335783 29408 tablet_bootstrap.cc:492] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.335807 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":47098}
I20260809 07:58:48.335966 29408 ts_tablet_manager.cc:1403] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.002s
I20260809 07:58:48.336086 29405 tablet_bootstrap.cc:492] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.336289 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.046s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":45883}
I20260809 07:58:48.336675 29408 raft_consensus.cc:359] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.336673 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.005s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":161,"lbm_writes_lt_1ms":4,"mutex_wait_us":41654,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.336859 29408 raft_consensus.cc:385] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.336937 29408 raft_consensus.cc:740] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.337189 29408 consensus_queue.cc:260] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.337862 29405 tablet_bootstrap.cc:654] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.338223 29408 ts_tablet_manager.cc:1434] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:48.338557 29408 tablet_bootstrap.cc:492] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.340313 29408 tablet_bootstrap.cc:654] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.337188 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.072s user 0.008s sys 0.005s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":19698,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":84,"dirs.run_cpu_time_us":224,"dirs.run_wall_time_us":1063,"lbm_read_time_us":564,"lbm_reads_lt_1ms":23,"lbm_write_time_us":3479,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"mutex_wait_us":20071,"peak_mem_usage":32768,"reinsert_count":0,"update_count":22}
I20260809 07:58:48.345613 29211 raft_consensus.cc:697] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:48.346006 29211 consensus_queue.cc:237] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.347478 29263 tablet_bootstrap.cc:492] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.347679 29263 ts_tablet_manager.cc:1403] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.002s
I20260809 07:58:48.348608 29263 raft_consensus.cc:359] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.348879 29263 raft_consensus.cc:385] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.349004 29263 raft_consensus.cc:740] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.349301 29263 consensus_queue.cc:260] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.350246 29263 ts_tablet_manager.cc:1434] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 07:58:48.350871 29263 tablet_bootstrap.cc:492] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.353276 29263 tablet_bootstrap.cc:654] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.326390 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.058s user 0.014s sys 0.001s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":20033,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":84,"dirs.run_cpu_time_us":193,"dirs.run_wall_time_us":1369,"lbm_read_time_us":613,"lbm_reads_lt_1ms":29,"lbm_write_time_us":451,"lbm_writes_lt_1ms":9,"mutex_wait_us":29476,"peak_mem_usage":32768,"reinsert_count":0,"update_count":23}
I20260809 07:58:48.364595 28151 catalog_manager.cc:5719] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:48.376392 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.376789 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.377077 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.377425 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.377461 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.377874 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.378135 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.378499 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.378746 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.379562 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.379897 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.379928 29211 raft_consensus.cc:493] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.380151 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.380334 29211 raft_consensus.cc:515] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.380877 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.381222 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.381601 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.382212 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ffe5276280a42e3830a3b6551950020" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.382493 28461 raft_consensus.cc:2468] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.382530 29211 leader_election.cc:290] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.382813 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ffe5276280a42e3830a3b6551950020" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.382930 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.382903 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":158,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.383128 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
W20260809 07:58:48.383450 28740 leader_election.cc:343] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:48.383709 28740 leader_election.cc:304] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:48.387981 29211 raft_consensus.cc:2804] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.388175 29211 raft_consensus.cc:493] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.388372 29211 raft_consensus.cc:3060] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.393903 29211 raft_consensus.cc:515] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.395733 29211 leader_election.cc:290] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.396209 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ffe5276280a42e3830a3b6551950020" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.396701 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.005s sys 0.002s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":185,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.396874 29405 tablet_bootstrap.cc:492] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.397039 29405 ts_tablet_manager.cc:1403] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.061s user 0.014s sys 0.032s
I20260809 07:58:48.397044 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ffe5276280a42e3830a3b6551950020" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
W20260809 07:58:48.397181 28740 leader_election.cc:343] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:48.397249 28461 raft_consensus.cc:3060] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.397974 29405 raft_consensus.cc:359] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.398188 29405 raft_consensus.cc:385] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.398262 29405 raft_consensus.cc:740] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.399857 29408 tablet_bootstrap.cc:492] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.400002 29408 ts_tablet_manager.cc:1403] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.062s user 0.022s sys 0.022s
I20260809 07:58:48.398500 29405 consensus_queue.cc:260] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.400686 28461 raft_consensus.cc:2468] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.400815 29405 ts_tablet_manager.cc:1434] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:48.401316 28740 leader_election.cc:304] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:48.401723 29405 tablet_bootstrap.cc:492] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.401849 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":670,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":48,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":4020,"lbm_read_time_us":207,"lbm_reads_lt_1ms":8,"lbm_write_time_us":186,"lbm_writes_lt_1ms":4,"mutex_wait_us":5341,"reinsert_count":0,"update_count":2}
I20260809 07:58:48.403573 29405 tablet_bootstrap.cc:654] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.405560 29408 raft_consensus.cc:359] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.405750 29408 raft_consensus.cc:385] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.405869 29408 raft_consensus.cc:740] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.406895 29405 tablet_bootstrap.cc:492] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.407032 29405 ts_tablet_manager.cc:1403] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260809 07:58:48.407282 29408 consensus_queue.cc:260] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.407688 29405 raft_consensus.cc:359] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.407827 29405 raft_consensus.cc:385] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.407892 29405 raft_consensus.cc:740] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.408066 29405 consensus_queue.cc:260] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.409142 29405 ts_tablet_manager.cc:1434] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:48.409399 29405 tablet_bootstrap.cc:492] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.409749 29408 ts_tablet_manager.cc:1434] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260809 07:58:48.410065 29408 tablet_bootstrap.cc:492] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.410944 29405 tablet_bootstrap.cc:654] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.411674 28961 consensus_queue.cc:1048] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.412632 29408 tablet_bootstrap.cc:654] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.413208 29474 raft_consensus.cc:2804] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.413381 29474 raft_consensus.cc:697] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:48.413455 29211 raft_consensus.cc:493] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.413671 29211 raft_consensus.cc:515] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.413825 29474 consensus_queue.cc:237] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.414716 29211 leader_election.cc:290] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.415618 29263 tablet_bootstrap.cc:492] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.415653 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984b64e2ac0441508af6773002acdb97" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.415805 29263 ts_tablet_manager.cc:1403] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.017s
I20260809 07:58:48.415934 28648 raft_consensus.cc:2393] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a091b20dbc4ed2b3bf6c960df964c5 in current term 1: Already voted for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in this term.
I20260809 07:58:48.416806 29263 raft_consensus.cc:359] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.417065 29263 raft_consensus.cc:385] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.417609 29263 raft_consensus.cc:740] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.417877 29263 consensus_queue.cc:260] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.418511 29263 ts_tablet_manager.cc:1434] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:48.419170 29263 tablet_bootstrap.cc:492] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.419250 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984b64e2ac0441508af6773002acdb97" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.419793 29408 tablet_bootstrap.cc:492] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.419821 28740 leader_election.cc:304] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:48.419941 29408 ts_tablet_manager.cc:1403] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260809 07:58:48.420147 29211 raft_consensus.cc:3060] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.420647 29408 raft_consensus.cc:359] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.420836 29408 raft_consensus.cc:385] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.420922 29408 raft_consensus.cc:740] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.421180 29408 consensus_queue.cc:260] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.421789 29408 ts_tablet_manager.cc:1434] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:48.422097 29408 tablet_bootstrap.cc:492] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.423736 29408 tablet_bootstrap.cc:654] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.423974 29211 raft_consensus.cc:2749] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:48.425545 29263 tablet_bootstrap.cc:654] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.427294 29405 tablet_bootstrap.cc:492] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.427443 29405 ts_tablet_manager.cc:1403] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.007s
I20260809 07:58:48.428062 29405 raft_consensus.cc:359] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.428246 29405 raft_consensus.cc:385] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.428314 29405 raft_consensus.cc:740] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.428543 29405 consensus_queue.cc:260] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.429544 29405 ts_tablet_manager.cc:1434] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:48.429853 29405 tablet_bootstrap.cc:492] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.431509 29408 tablet_bootstrap.cc:492] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.431639 29408 ts_tablet_manager.cc:1403] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260809 07:58:48.431936 29405 tablet_bootstrap.cc:654] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.432355 29408 raft_consensus.cc:359] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.432545 29408 raft_consensus.cc:385] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.432641 29408 raft_consensus.cc:740] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.432847 29408 consensus_queue.cc:260] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.433461 29408 ts_tablet_manager.cc:1434] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:48.433703 29408 tablet_bootstrap.cc:492] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.434127 29263 tablet_bootstrap.cc:492] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.434281 29263 ts_tablet_manager.cc:1403] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.003s
I20260809 07:58:48.435545 29408 tablet_bootstrap.cc:654] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.435482 29263 raft_consensus.cc:359] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.435695 29263 raft_consensus.cc:385] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.435807 29263 raft_consensus.cc:740] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.436113 29263 consensus_queue.cc:260] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.436791 29263 ts_tablet_manager.cc:1434] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 07:58:48.437152 29263 tablet_bootstrap.cc:492] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.439035 29263 tablet_bootstrap.cc:654] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.441814 29408 tablet_bootstrap.cc:492] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.441977 29408 ts_tablet_manager.cc:1403] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 07:58:48.442777 29408 raft_consensus.cc:359] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.442752 28153 catalog_manager.cc:5719] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:48.443039 29408 raft_consensus.cc:385] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.443112 29408 raft_consensus.cc:740] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.443352 29408 consensus_queue.cc:260] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.443887 29408 ts_tablet_manager.cc:1434] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:48.444159 29405 tablet_bootstrap.cc:492] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.444288 29405 ts_tablet_manager.cc:1403] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.007s
I20260809 07:58:48.444476 29263 tablet_bootstrap.cc:492] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:48.444633 29263 ts_tablet_manager.cc:1403] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260809 07:58:48.445065 29405 raft_consensus.cc:359] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.445238 29405 raft_consensus.cc:385] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.445369 29263 raft_consensus.cc:359] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.445461 28961 raft_consensus.cc:493] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.445554 29263 raft_consensus.cc:385] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.445642 29263 raft_consensus.cc:740] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.445637 28961 raft_consensus.cc:515] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.445887 29263 consensus_queue.cc:260] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.446327 28961 leader_election.cc:290] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.446401 29263 ts_tablet_manager.cc:1434] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:48.446918 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01bd743cf4eb439eb0e7b86655b630ec" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.447124 28461 raft_consensus.cc:2468] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:48.447113 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01bd743cf4eb439eb0e7b86655b630ec" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.447284 28820 raft_consensus.cc:2468] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:48.447497 29405 raft_consensus.cc:740] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.447700 29405 consensus_queue.cc:260] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.447834 28565 leader_election.cc:304] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.448232 29405 ts_tablet_manager.cc:1434] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 07:58:48.448371 28961 raft_consensus.cc:2804] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.448454 28961 raft_consensus.cc:493] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.448514 29405 tablet_bootstrap.cc:492] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.448549 28961 raft_consensus.cc:3060] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.450238 29405 tablet_bootstrap.cc:654] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.450960 28961 raft_consensus.cc:515] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.452124 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01bd743cf4eb439eb0e7b86655b630ec" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:48.452312 28820 raft_consensus.cc:3060] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.452322 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01bd743cf4eb439eb0e7b86655b630ec" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:48.452527 28461 raft_consensus.cc:3060] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.455173 28461 raft_consensus.cc:2468] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:48.455260 28820 raft_consensus.cc:2468] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:48.455751 28961 leader_election.cc:290] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.456030 28565 leader_election.cc:304] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.456524 28961 raft_consensus.cc:2804] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.456764 28961 raft_consensus.cc:697] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:48.457010 28961 consensus_queue.cc:237] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.459290 29405 tablet_bootstrap.cc:492] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.459455 29405 ts_tablet_manager.cc:1403] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.003s
I20260809 07:58:48.461117 29451 consensus_queue.cc:1048] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.461114 29405 raft_consensus.cc:359] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.461467 29405 raft_consensus.cc:385] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.461586 29405 raft_consensus.cc:740] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.461899 29405 consensus_queue.cc:260] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.462752 29405 ts_tablet_manager.cc:1434] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:48.463111 29405 tablet_bootstrap.cc:492] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.465821 29405 tablet_bootstrap.cc:654] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.466777 29451 raft_consensus.cc:493] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.466951 29451 raft_consensus.cc:515] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.467768 29451 leader_election.cc:290] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.468531 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d7600198d40e3a3470352eb70b2c6" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.468704 28820 raft_consensus.cc:2468] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.468696 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.469048 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.469403 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.469732 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.471990 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.472363 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.472635 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.472755 29405 tablet_bootstrap.cc:492] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.472903 29405 ts_tablet_manager.cc:1403] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260809 07:58:48.473187 28382 leader_election.cc:304] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.473660 29405 raft_consensus.cc:359] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.475176 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d7600198d40e3a3470352eb70b2c6" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.475400 28647 raft_consensus.cc:2468] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.475919 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.473830 29405 raft_consensus.cc:385] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.476191 29405 raft_consensus.cc:740] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.476218 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.476501 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.476857 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.477108 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.477468 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.479423 29451 raft_consensus.cc:2804] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.479552 29451 raft_consensus.cc:493] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.479645 29451 raft_consensus.cc:3060] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.486634 29451 raft_consensus.cc:515] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.487000 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":136,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.487461 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.487807 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.488365 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.490015 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.005s sys 0.002s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":670,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":143,"lbm_read_time_us":121,"lbm_reads_lt_1ms":8,"lbm_write_time_us":131,"lbm_writes_lt_1ms":4,"mutex_wait_us":3864,"reinsert_count":0,"update_count":2}
I20260809 07:58:48.490542 29405 consensus_queue.cc:260] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.492107 29405 ts_tablet_manager.cc:1434] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20260809 07:58:48.492470 29405 tablet_bootstrap.cc:492] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.494796 29405 tablet_bootstrap.cc:654] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.504391 29405 tablet_bootstrap.cc:492] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.504630 29405 ts_tablet_manager.cc:1403] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.001s
I20260809 07:58:48.505391 29405 raft_consensus.cc:359] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.505716 29405 raft_consensus.cc:385] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.505846 29405 raft_consensus.cc:740] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.506126 29405 consensus_queue.cc:260] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.506736 29405 ts_tablet_manager.cc:1434] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:58:48.507184 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.002s sys 0.004s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":195,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.508510 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d7600198d40e3a3470352eb70b2c6" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.508759 28648 raft_consensus.cc:3060] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.511057 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":712,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":46,"lbm_read_time_us":233,"lbm_reads_lt_1ms":8,"lbm_write_time_us":221,"lbm_writes_lt_1ms":4,"mutex_wait_us":14612,"reinsert_count":0,"update_count":3}
I20260809 07:58:48.512528 28648 raft_consensus.cc:2468] T 160d7600198d40e3a3470352eb70b2c6 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.514422 29474 consensus_queue.cc:1048] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.514909 28380 leader_election.cc:304] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.515312 29314 raft_consensus.cc:2804] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.515511 29314 raft_consensus.cc:697] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:48.515849 29314 consensus_queue.cc:237] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.516444 29451 leader_election.cc:290] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.516600 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d7600198d40e3a3470352eb70b2c6" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:48.516865 28820 raft_consensus.cc:3060] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.519709 28820 raft_consensus.cc:2468] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.525463 29453 consensus_queue.cc:1048] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.525960 29211 consensus_queue.cc:1048] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.531487 28151 catalog_manager.cc:5719] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.542667 28153 catalog_manager.cc:5719] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.542908 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.543226 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.543463 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.543792 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.544072 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.544409 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.544704 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.545015 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.545257 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.546753 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.546963 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.547252 29001 raft_consensus.cc:493] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.547384 29001 raft_consensus.cc:515] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.547974 29001 leader_election.cc:290] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.548277 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.548534 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.549141 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.549508 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.549714 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.549998 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.550181 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907d3db7702546e58c219fc805be8f7c" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.550419 28648 raft_consensus.cc:2468] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.550930 28740 leader_election.cc:304] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.551244 29001 raft_consensus.cc:2804] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.551375 29001 raft_consensus.cc:493] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.551456 29001 raft_consensus.cc:3060] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.551589 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907d3db7702546e58c219fc805be8f7c" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.551757 28461 raft_consensus.cc:2468] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.554400 29001 raft_consensus.cc:515] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.554780 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.001s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":138,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.554875 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":9287,"spinlock_wait_cycles":1280}
I20260809 07:58:48.555452 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907d3db7702546e58c219fc805be8f7c" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:48.555640 28461 raft_consensus.cc:3060] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.556183 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907d3db7702546e58c219fc805be8f7c" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.556411 28648 raft_consensus.cc:3060] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.548220 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":130,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.557966 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.005s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":735,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":165,"lbm_reads_lt_1ms":8,"lbm_write_time_us":175,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":4}
I20260809 07:58:48.558321 29453 raft_consensus.cc:493] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.558498 29453 raft_consensus.cc:515] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.558594 28461 raft_consensus.cc:2468] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.559080 28740 leader_election.cc:304] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.559295 29453 leader_election.cc:290] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.559576 29211 raft_consensus.cc:2804] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.559664 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c26d23e8037447c9fdfee1c82f54790" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.559736 29211 raft_consensus.cc:697] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:48.559921 28820 raft_consensus.cc:2468] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.560127 29211 consensus_queue.cc:237] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.560660 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c26d23e8037447c9fdfee1c82f54790" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.560871 28647 raft_consensus.cc:2468] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.561268 28380 leader_election.cc:304] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.561652 29453 raft_consensus.cc:2804] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.561781 29453 raft_consensus.cc:493] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.561928 29453 raft_consensus.cc:3060] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.561697 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":695,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":53,"lbm_read_time_us":163,"lbm_reads_lt_1ms":8,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"mutex_wait_us":5273,"reinsert_count":0,"update_count":3}
I20260809 07:58:48.564002 28648 raft_consensus.cc:2468] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.565073 29001 leader_election.cc:290] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.565009 29453 raft_consensus.cc:515] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.566080 29453 leader_election.cc:290] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.566985 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c26d23e8037447c9fdfee1c82f54790" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.567468 28648 raft_consensus.cc:3060] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.567459 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c26d23e8037447c9fdfee1c82f54790" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:48.567673 28820 raft_consensus.cc:3060] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.569674 28820 raft_consensus.cc:2468] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.570120 28382 leader_election.cc:304] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.570410 29453 raft_consensus.cc:2804] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.570588 28648 raft_consensus.cc:2468] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.571223 29453 raft_consensus.cc:697] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:48.571573 29453 consensus_queue.cc:237] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.572765 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.573006 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.573289 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.573673 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.573915 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.574227 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.574453 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.574857 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.575330 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.575702 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.576033 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.577656 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.577967 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.578203 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.578426 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.578776 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.579229 28153 catalog_manager.cc:5719] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:48.582693 28151 catalog_manager.cc:5719] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.592701 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.593026 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.586230 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.004s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":190,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.594081 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.594262 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.005s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":768,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":40,"lbm_read_time_us":188,"lbm_reads_lt_1ms":8,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"mutex_wait_us":6518,"reinsert_count":0,"update_count":5}
I20260809 07:58:48.595598 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.595628 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.595860 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.597863 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":723,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":41,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":151,"lbm_writes_lt_1ms":4,"mutex_wait_us":12468,"reinsert_count":0,"update_count":4}
I20260809 07:58:48.598379 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.598469 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.004s sys 0.000s Metrics: {"bytes_written":466,"delete_count":0,"lbm_write_time_us":130,"lbm_writes_lt_1ms":4,"mutex_wait_us":1266,"reinsert_count":0,"update_count":2}
I20260809 07:58:48.599197 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.600610 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.006s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":239,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.601583 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":5331}
I20260809 07:58:48.601543 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":466,"delete_count":0,"lbm_write_time_us":137,"lbm_writes_lt_1ms":4,"mutex_wait_us":1762,"reinsert_count":0,"update_count":2}
I20260809 07:58:48.603926 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":10551}
I20260809 07:58:48.603896 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":830,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":54,"lbm_read_time_us":119,"lbm_reads_lt_1ms":8,"lbm_write_time_us":111,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":7}
I20260809 07:58:48.606485 29453 consensus_queue.cc:1048] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.607095 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":779,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":49,"lbm_read_time_us":134,"lbm_reads_lt_1ms":8,"lbm_write_time_us":145,"lbm_writes_lt_1ms":4,"mutex_wait_us":2165,"reinsert_count":0,"update_count":6}
I20260809 07:58:48.611244 29014 consensus_queue.cc:1048] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.614053 29211 raft_consensus.cc:493] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.614228 29211 raft_consensus.cc:515] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.614916 29211 leader_election.cc:290] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.615599 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a22d70306946e3812d58884686f10c" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.616070 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a22d70306946e3812d58884686f10c" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.616284 28461 raft_consensus.cc:2468] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.615836 28648 raft_consensus.cc:2468] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.617120 28740 leader_election.cc:304] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.617419 29211 raft_consensus.cc:2804] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.617591 29211 raft_consensus.cc:493] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.617699 29211 raft_consensus.cc:3060] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.620217 29211 raft_consensus.cc:515] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.620787 29211 leader_election.cc:290] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.621169 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a22d70306946e3812d58884686f10c" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:48.621258 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a22d70306946e3812d58884686f10c" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.621425 28461 raft_consensus.cc:3060] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.621495 28648 raft_consensus.cc:3060] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.623267 28461 raft_consensus.cc:2468] T 70a22d70306946e3812d58884686f10c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.623458 28648 raft_consensus.cc:2468] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.623662 28740 leader_election.cc:304] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.623914 29211 raft_consensus.cc:2804] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.624063 29211 raft_consensus.cc:697] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:48.624393 29211 consensus_queue.cc:237] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.626806 28149 catalog_manager.cc:5719] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:48.627075 28961 consensus_queue.cc:1048] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.633036 29029 consensus_queue.cc:1048] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.635494 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.635794 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.636225 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.636516 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.636705 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.636927 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.637228 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.637634 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.638393 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.638729 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.639008 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.639241 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.639598 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.639844 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.640097 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.640759 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.641026 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.641399 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.644250 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.005s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":141,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.644407 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":151,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.647291 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.002s sys 0.002s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":843,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":66,"lbm_read_time_us":175,"lbm_reads_lt_1ms":8,"lbm_write_time_us":124,"lbm_writes_lt_1ms":4,"mutex_wait_us":4260,"reinsert_count":0,"update_count":8}
I20260809 07:58:48.649148 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.002s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":800,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":65,"lbm_read_time_us":160,"lbm_reads_lt_1ms":8,"lbm_write_time_us":124,"lbm_writes_lt_1ms":4,"mutex_wait_us":3362,"reinsert_count":0,"update_count":7}
I20260809 07:58:48.666038 29029 raft_consensus.cc:493] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.666205 29029 raft_consensus.cc:515] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.666783 29029 leader_election.cc:290] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.667433 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ffe5276280a42e3830a3b6551950020" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.667460 29014 raft_consensus.cc:493] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.667449 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ffe5276280a42e3830a3b6551950020" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.667724 28461 raft_consensus.cc:2393] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24137fd377124538a31e92f3b6df0673 in current term 1: Already voted for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in this term.
I20260809 07:58:48.667657 29014 raft_consensus.cc:515] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.668099 28565 leader_election.cc:304] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:48.668344 29014 leader_election.cc:290] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.668438 29029 raft_consensus.cc:3060] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.668737 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe33c93dd945feb415cf918022f0d9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.668939 28648 raft_consensus.cc:2468] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.669040 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe33c93dd945feb415cf918022f0d9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:48.669262 28820 raft_consensus.cc:2468] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:48.669385 28380 leader_election.cc:304] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.669685 29014 raft_consensus.cc:2804] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.669806 29014 raft_consensus.cc:493] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.669914 29014 raft_consensus.cc:3060] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.671080 29029 raft_consensus.cc:2749] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:48.671933 29014 raft_consensus.cc:515] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.672516 29014 leader_election.cc:290] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:48.672900 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe33c93dd945feb415cf918022f0d9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.672987 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dbe33c93dd945feb415cf918022f0d9" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:48.673080 28648 raft_consensus.cc:3060] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.673218 28820 raft_consensus.cc:3060] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.674971 28648 raft_consensus.cc:2468] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.675072 28820 raft_consensus.cc:2468] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:48.675427 28380 leader_election.cc:304] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:48.675732 29014 raft_consensus.cc:2804] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.675900 29014 raft_consensus.cc:697] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:48.676159 29014 consensus_queue.cc:237] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.678898 28151 catalog_manager.cc:5719] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:48.688746 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.689226 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.689553 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.689908 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.690193 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.690446 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.690804 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.691121 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.691506 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.691862 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.692116 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.692560 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.693955 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.004s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":170,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.695499 29211 raft_consensus.cc:493] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:48.695643 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":135,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.695737 29211 raft_consensus.cc:515] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.696529 29211 leader_election.cc:290] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.697274 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00164db27d9243e08c9b667d92c89297" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:48.697325 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00164db27d9243e08c9b667d92c89297" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:48.697535 28461 raft_consensus.cc:2468] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.697738 28648 raft_consensus.cc:2468] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:48.698017 28740 leader_election.cc:304] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.698323 29211 raft_consensus.cc:2804] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:48.698439 29211 raft_consensus.cc:493] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:48.698566 29211 raft_consensus.cc:3060] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.700914 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.002s sys 0.002s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":864,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":75,"lbm_read_time_us":99,"lbm_reads_lt_1ms":8,"lbm_write_time_us":177,"lbm_writes_lt_1ms":4,"mutex_wait_us":3776,"reinsert_count":0,"update_count":9}
I20260809 07:58:48.701409 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":821,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":41,"lbm_read_time_us":123,"lbm_reads_lt_1ms":8,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"mutex_wait_us":3206,"reinsert_count":0,"update_count":8}
I20260809 07:58:48.701582 29211 raft_consensus.cc:515] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.702203 29211 leader_election.cc:290] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:48.702736 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00164db27d9243e08c9b667d92c89297" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:48.702754 28461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00164db27d9243e08c9b667d92c89297" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:48.702930 28461 raft_consensus.cc:3060] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.702947 28648 raft_consensus.cc:3060] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:48.705005 28648 raft_consensus.cc:2468] T 00164db27d9243e08c9b667d92c89297 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.704986 28461 raft_consensus.cc:2468] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:48.705473 28740 leader_election.cc:304] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:48.705753 29211 raft_consensus.cc:2804] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:48.706014 29211 raft_consensus.cc:697] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:48.706342 29211 consensus_queue.cc:237] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.706720 29014 consensus_queue.cc:1048] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.709483 28151 catalog_manager.cc:5719] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:48.722451 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.722762 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.723093 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.723377 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.723649 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.723798 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.723793 29451 consensus_queue.cc:1048] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.724021 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.724427 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.731117 28153 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51156:
table { table_name: "table-3" } modify_external_catalogs: true
I20260809 07:58:48.731379 28153 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51156:
table { table_name: "table-3" } modify_external_catalogs: true
I20260809 07:58:48.732837 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.004s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":212,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.737258 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.737641 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.737936 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.738260 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.738579 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.739379 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.740025 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.002s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":843,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":2920,"lbm_read_time_us":158,"lbm_reads_lt_1ms":8,"lbm_write_time_us":179,"lbm_writes_lt_1ms":4,"mutex_wait_us":4791,"reinsert_count":0,"update_count":9}
I20260809 07:58:48.742364 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.001s sys 0.002s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":179,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:48.745628 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.743690 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.745934 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.746019 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.746239 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.746332 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.746733 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.747006 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.747370 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.331841
I20260809 07:58:48.747663 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.331841
I20260809 07:58:48.748155 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.748500 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:48.748765 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.750538 29451 consensus_queue.cc:1048] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.750749 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet e73d79f178374da8b18a76cb1e350939
I20260809 07:58:48.751370 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 43464de37a144d689c41c6f37d20f6f0
I20260809 07:58:48.751658 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet e73d79f178374da8b18a76cb1e350939 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.751838 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 3ee878d7fd574bac94938eb05a9612c9
I20260809 07:58:48.752427 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet ffcb1553e25545ebb774d72295e65fbf
I20260809 07:58:48.752430 28617 tablet_service.cc:1558] Processing DeleteTablet for tablet e73d79f178374da8b18a76cb1e350939 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.752436 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 43464de37a144d689c41c6f37d20f6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.752914 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet cda618ed0c4640839f31835d02f906c9
I20260809 07:58:48.752950 28794 tablet_service.cc:1558] Processing DeleteTablet for tablet e73d79f178374da8b18a76cb1e350939 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.753468 28795 tablet_service.cc:1558] Processing DeleteTablet for tablet 43464de37a144d689c41c6f37d20f6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.753518 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 40faafbbf8c646aea67e679524df2cc5
I20260809 07:58:48.753753 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 43464de37a144d689c41c6f37d20f6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.754002 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 56e4487e4b164b49903280601c1b835d
I20260809 07:58:48.754052 28620 tablet_service.cc:1558] Processing DeleteTablet for tablet cda618ed0c4640839f31835d02f906c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.754148 28782 tablet_service.cc:1558] Processing DeleteTablet for tablet e73d79f178374da8b18a76cb1e350939 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.754065 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet ffcb1553e25545ebb774d72295e65fbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.754364 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ee878d7fd574bac94938eb05a9612c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.754473 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 984b64e2ac0441508af6773002acdb97
I20260809 07:58:48.754825 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ee878d7fd574bac94938eb05a9612c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.755036 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet ffcb1553e25545ebb774d72295e65fbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.755136 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet cda618ed0c4640839f31835d02f906c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.755259 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 0dbe33c93dd945feb415cf918022f0d9
I20260809 07:58:48.755745 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 907d3db7702546e58c219fc805be8f7c
I20260809 07:58:48.756239 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 01bd743cf4eb439eb0e7b86655b630ec
I20260809 07:58:48.756582 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 0dbe33c93dd945feb415cf918022f0d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.756824 28620 tablet_service.cc:1558] Processing DeleteTablet for tablet 984b64e2ac0441508af6773002acdb97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.756901 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet cda618ed0c4640839f31835d02f906c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.756947 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 5c26d23e8037447c9fdfee1c82f54790
I20260809 07:58:48.757047 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e4487e4b164b49903280601c1b835d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.757200 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet 40faafbbf8c646aea67e679524df2cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.757272 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 40faafbbf8c646aea67e679524df2cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.757444 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e4487e4b164b49903280601c1b835d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.757601 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 7ffe5276280a42e3830a3b6551950020
I20260809 07:58:48.757685 28787 tablet_service.cc:1558] Processing DeleteTablet for tablet 0dbe33c93dd945feb415cf918022f0d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.757854 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e4487e4b164b49903280601c1b835d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.757896 28798 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c26d23e8037447c9fdfee1c82f54790 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.758111 28795 tablet_service.cc:1558] Processing DeleteTablet for tablet ffcb1553e25545ebb774d72295e65fbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.758104 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet 984b64e2ac0441508af6773002acdb97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.758213 28784 tablet_service.cc:1558] Processing DeleteTablet for tablet 907d3db7702546e58c219fc805be8f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.758314 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 01bd743cf4eb439eb0e7b86655b630ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.758324 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 0dbe33c93dd945feb415cf918022f0d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.758477 28782 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ee878d7fd574bac94938eb05a9612c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.757905 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet 984b64e2ac0441508af6773002acdb97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.758749 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 40faafbbf8c646aea67e679524df2cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.758975 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 70a22d70306946e3812d58884686f10c
I20260809 07:58:48.759275 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.004s sys 0.001s Metrics: {"bytes_written":926,"delete_count":0,"lbm_write_time_us":157,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260809 07:58:48.759635 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 00164db27d9243e08c9b667d92c89297
I20260809 07:58:48.759797 28603 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ffe5276280a42e3830a3b6551950020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.760026 28620 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c26d23e8037447c9fdfee1c82f54790 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.760249 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet 01bd743cf4eb439eb0e7b86655b630ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.760334 28153 catalog_manager.cc:6006] T 00000000000000000000000000000000 P 16fe06ae10064d4b894ac1d45c6ca3f6: Sending DeleteTablet for 3 replicas of tablet 160d7600198d40e3a3470352eb70b2c6
I20260809 07:58:48.760468 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 907d3db7702546e58c219fc805be8f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.761018 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 01bd743cf4eb439eb0e7b86655b630ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.761200 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 907d3db7702546e58c219fc805be8f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.761313 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c26d23e8037447c9fdfee1c82f54790 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.761603 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ffe5276280a42e3830a3b6551950020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.762292 28602 tablet_service.cc:1558] Processing DeleteTablet for tablet 70a22d70306946e3812d58884686f10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.762470 28614 tablet_service.cc:1558] Processing DeleteTablet for tablet 00164db27d9243e08c9b667d92c89297 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.762691 28620 tablet_service.cc:1558] Processing DeleteTablet for tablet 160d7600198d40e3a3470352eb70b2c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:40438
I20260809 07:58:48.763031 28429 tablet_service.cc:1558] Processing DeleteTablet for tablet 00164db27d9243e08c9b667d92c89297 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.763025 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 160d7600198d40e3a3470352eb70b2c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.763266 28432 tablet_service.cc:1558] Processing DeleteTablet for tablet 70a22d70306946e3812d58884686f10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:59804
I20260809 07:58:48.763926 28797 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ffe5276280a42e3830a3b6551950020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.763981 28784 tablet_service.cc:1558] Processing DeleteTablet for tablet 160d7600198d40e3a3470352eb70b2c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.764175 28788 tablet_service.cc:1558] Processing DeleteTablet for tablet 00164db27d9243e08c9b667d92c89297 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.764407 28782 tablet_service.cc:1558] Processing DeleteTablet for tablet 70a22d70306946e3812d58884686f10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:58:48 UTC) from {username='slave'} at 127.0.0.1:58024
I20260809 07:58:48.766899 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.006s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":891,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":144,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4,"lbm_write_time_us":154,"lbm_writes_lt_1ms":4,"mutex_wait_us":2706,"reinsert_count":0,"update_count":10}
I20260809 07:58:48.767613 29526 tablet_replica.cc:333] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:48.767916 29526 raft_consensus.cc:2243] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:48.768182 29526 raft_consensus.cc:2272] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.769253 29525 tablet_replica.cc:333] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:48.769505 29525 raft_consensus.cc:2243] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:48.769625 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.334034
I20260809 07:58:48.770191 29525 raft_consensus.cc:2272] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.771428 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.013s sys 0.001s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":23875,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":80,"lbm_read_time_us":587,"lbm_reads_lt_1ms":27,"lbm_write_time_us":424,"lbm_writes_lt_1ms":9,"mutex_wait_us":4495,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":846080,"update_count":26}
I20260809 07:58:48.772224 28153 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51156:
name: "table-4"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
I20260809 07:58:48.772818 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":26167}
I20260809 07:58:48.773097 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":23754}
W20260809 07:58:48.773361 28153 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:58:48.772763 29525 ts_tablet_manager.cc:1916] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.776831 29527 tablet_replica.cc:333] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:48.777087 29527 raft_consensus.cc:2243] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:48.777370 29527 raft_consensus.cc:2272] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.778532 29526 ts_tablet_manager.cc:1916] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.778594 29340 consensus_queue.cc:1048] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.779067 29527 ts_tablet_manager.cc:1916] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.786981 29525 ts_tablet_manager.cc:1929] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.787902 29526 ts_tablet_manager.cc:1929] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.788096 29526 log.cc:1199] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/e73d79f178374da8b18a76cb1e350939
I20260809 07:58:48.788690 29526 ts_tablet_manager.cc:1950] T e73d79f178374da8b18a76cb1e350939 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:48.789937 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet e73d79f178374da8b18a76cb1e350939 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:48.790464 29526 tablet_replica.cc:333] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:48.790859 29526 raft_consensus.cc:2243] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:48.791090 29526 raft_consensus.cc:2272] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.792703 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.797560 29526 ts_tablet_manager.cc:1916] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.799114 29527 ts_tablet_manager.cc:1929] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.799278 29527 log.cc:1199] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/e73d79f178374da8b18a76cb1e350939
I20260809 07:58:48.799489 29525 log.cc:1199] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/e73d79f178374da8b18a76cb1e350939
I20260809 07:58:48.799986 29525 ts_tablet_manager.cc:1950] T e73d79f178374da8b18a76cb1e350939 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:48.800953 29525 tablet_replica.cc:333] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:48.801198 29525 raft_consensus.cc:2243] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:48.801601 29525 raft_consensus.cc:2272] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.801854 29527 ts_tablet_manager.cc:1950] T e73d79f178374da8b18a76cb1e350939 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:48.802966 29527 tablet_replica.cc:333] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:48.803201 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet e73d79f178374da8b18a76cb1e350939 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:48.803536 29527 raft_consensus.cc:2243] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:48.803746 29527 raft_consensus.cc:2272] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.804858 29526 ts_tablet_manager.cc:1929] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.805042 29526 log.cc:1199] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/43464de37a144d689c41c6f37d20f6f0
I20260809 07:58:48.805320 29525 ts_tablet_manager.cc:1916] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.805580 29526 ts_tablet_manager.cc:1950] T 43464de37a144d689c41c6f37d20f6f0 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:48.805575 29527 ts_tablet_manager.cc:1916] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.806923 29526 tablet_replica.cc:333] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:48.807173 29526 raft_consensus.cc:2243] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:48.807091 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.008s sys 0.004s Metrics: {"bytes_written":6941,"cfile_init":1,"dirs.queue_time_us":67,"dirs.run_cpu_time_us":201,"dirs.run_wall_time_us":1164,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":995,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17,"spinlock_wait_cycles":11520}
I20260809 07:58:48.807502 29526 raft_consensus.cc:2272] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.808030 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:48.808499 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.260361
I20260809 07:58:48.808746 29526 ts_tablet_manager.cc:1916] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.808660 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260809 07:58:48.809157 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 43464de37a144d689c41c6f37d20f6f0 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:48.810901 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet e73d79f178374da8b18a76cb1e350939 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:48.813598 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.260361
I20260809 07:58:48.814043 29525 ts_tablet_manager.cc:1929] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.814244 29525 log.cc:1199] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/43464de37a144d689c41c6f37d20f6f0
I20260809 07:58:48.814828 29525 ts_tablet_manager.cc:1950] T 43464de37a144d689c41c6f37d20f6f0 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:48.815991 29525 tablet_replica.cc:333] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:48.816244 29525 raft_consensus.cc:2243] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:48.816496 29525 raft_consensus.cc:2272] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.817293 29526 ts_tablet_manager.cc:1929] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.817524 29526 log.cc:1199] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/cda618ed0c4640839f31835d02f906c9
I20260809 07:58:48.817865 29525 ts_tablet_manager.cc:1916] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.818045 29526 ts_tablet_manager.cc:1950] T cda618ed0c4640839f31835d02f906c9 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:48.820048 29526 tablet_replica.cc:333] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:48.820267 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.820650 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.056s user 0.004s sys 0.001s Metrics: {"bytes_written":926,"delete_count":0,"lbm_write_time_us":139,"lbm_writes_lt_1ms":4,"mutex_wait_us":20861,"reinsert_count":0,"update_count":17}
I20260809 07:58:48.821403 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 43464de37a144d689c41c6f37d20f6f0 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:48.821818 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet cda618ed0c4640839f31835d02f906c9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:48.825181 29525 ts_tablet_manager.cc:1929] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.825354 29525 log.cc:1199] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/3ee878d7fd574bac94938eb05a9612c9
I20260809 07:58:48.825825 29525 ts_tablet_manager.cc:1950] T 3ee878d7fd574bac94938eb05a9612c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:48.826835 29525 tablet_replica.cc:333] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:48.827099 29525 raft_consensus.cc:2243] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:48.827308 29525 raft_consensus.cc:2272] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.827350 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 3ee878d7fd574bac94938eb05a9612c9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:48.828670 29525 ts_tablet_manager.cc:1916] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.834434 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.834965 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.835258 29525 ts_tablet_manager.cc:1929] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.835451 29525 log.cc:1199] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/ffcb1553e25545ebb774d72295e65fbf
I20260809 07:58:48.835686 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.836042 29525 ts_tablet_manager.cc:1950] T ffcb1553e25545ebb774d72295e65fbf P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:48.837422 28794 tablet_service.cc:1511] Processing CreateTablet for tablet 6bb7ff90e90944e29c8afad95fe45045 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.837925 28794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bb7ff90e90944e29c8afad95fe45045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.838068 28784 tablet_service.cc:1511] Processing CreateTablet for tablet a827ea5013564227907d06ef71b0dc58 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.838565 28784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a827ea5013564227907d06ef71b0dc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.839777 29527 ts_tablet_manager.cc:1929] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.839931 29527 log.cc:1199] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/43464de37a144d689c41c6f37d20f6f0
I20260809 07:58:48.840440 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 78b8280cd463421da5a4813a0df5d61a (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.840814 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b8280cd463421da5a4813a0df5d61a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.842341 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet ffcb1553e25545ebb774d72295e65fbf (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:48.842479 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.003s sys 0.002s Metrics: {"bytes_written":566,"delete_count":0,"lbm_write_time_us":180,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":9}
I20260809 07:58:48.842983 29211 consensus_queue.cc:1048] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.843151 28617 tablet_service.cc:1511] Processing CreateTablet for tablet 6bb7ff90e90944e29c8afad95fe45045 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.843608 28617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bb7ff90e90944e29c8afad95fe45045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.844077 28614 tablet_service.cc:1511] Processing CreateTablet for tablet 78b8280cd463421da5a4813a0df5d61a (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.844476 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b8280cd463421da5a4813a0df5d61a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.845604 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.845923 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:48.846364 28602 tablet_service.cc:1511] Processing CreateTablet for tablet a827ea5013564227907d06ef71b0dc58 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.846756 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a827ea5013564227907d06ef71b0dc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.849255 28603 tablet_service.cc:1511] Processing CreateTablet for tablet 84fc98b8ab394da1ae55ebd28e2a90d1 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.849510 29525 tablet_replica.cc:333] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:48.849565 29263 tablet_bootstrap.cc:492] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:48.849697 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84fc98b8ab394da1ae55ebd28e2a90d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.849875 29525 raft_consensus.cc:2243] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:48.850337 29525 raft_consensus.cc:2272] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.852124 29525 ts_tablet_manager.cc:1916] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.854023 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 6bb7ff90e90944e29c8afad95fe45045 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.854074 28432 tablet_service.cc:1511] Processing CreateTablet for tablet 58bf36691fc149519fb87fe4961b1cf5 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.854525 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bb7ff90e90944e29c8afad95fe45045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.854612 28429 tablet_service.cc:1511] Processing CreateTablet for tablet a827ea5013564227907d06ef71b0dc58 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.843569 28620 tablet_service.cc:1511] Processing CreateTablet for tablet 58bf36691fc149519fb87fe4961b1cf5 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.854972 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a827ea5013564227907d06ef71b0dc58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.854985 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58bf36691fc149519fb87fe4961b1cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.854417 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 78b8280cd463421da5a4813a0df5d61a (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.855346 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b8280cd463421da5a4813a0df5d61a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.857688 28437 tablet_service.cc:1511] Processing CreateTablet for tablet 84fc98b8ab394da1ae55ebd28e2a90d1 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.857669 29029 consensus_queue.cc:1048] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.858057 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84fc98b8ab394da1ae55ebd28e2a90d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.860894 29526 raft_consensus.cc:2243] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:48.861268 29526 raft_consensus.cc:2272] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:48.861761 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58bf36691fc149519fb87fe4961b1cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.865553 29408 tablet_bootstrap.cc:492] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.867859 29408 tablet_bootstrap.cc:654] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.873551 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.876001 29405 tablet_bootstrap.cc:492] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:48.877842 29405 tablet_bootstrap.cc:654] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.878940 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.005s sys 0.000s Metrics: {"bytes_written":529,"delete_count":0,"lbm_write_time_us":188,"lbm_writes_lt_1ms":4,"reinsert_count":0,"spinlock_wait_cycles":23387392,"update_count":7}
I20260809 07:58:48.886013 28788 tablet_service.cc:1511] Processing CreateTablet for tablet 84fc98b8ab394da1ae55ebd28e2a90d1 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.886584 28788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84fc98b8ab394da1ae55ebd28e2a90d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.889832 29211 consensus_queue.cc:1048] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.893924 28437 tablet_service.cc:1511] Processing CreateTablet for tablet f96b7f56c0144743854b2a78064dbc0d (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.894381 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f96b7f56c0144743854b2a78064dbc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.895151 28427 tablet_service.cc:1511] Processing CreateTablet for tablet c3903551936945ccbb1ab823751d3181 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.879565 28232 tablet.cc:2106] T 00000000000000000000000000000000 P d2e6b2cd9629422c8bcfa82f3cbad53a: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260809 07:58:48.896590 29526 ts_tablet_manager.cc:1916] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:48.899858 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3903551936945ccbb1ab823751d3181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.902419 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.088s user 0.010s sys 0.009s Metrics: {"dirs.queue_time_us":37326,"dirs.run_cpu_time_us":873,"dirs.run_wall_time_us":4923,"lbm_write_time_us":477,"lbm_writes_lt_1ms":7,"mutex_wait_us":23461,"num_input_rowsets":0,"thread_start_us":21360,"threads_started":1}
I20260809 07:58:48.904265 28797 tablet_service.cc:1511] Processing CreateTablet for tablet c3903551936945ccbb1ab823751d3181 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.904348 28795 tablet_service.cc:1511] Processing CreateTablet for tablet f96b7f56c0144743854b2a78064dbc0d (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.904675 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3903551936945ccbb1ab823751d3181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.904950 28795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f96b7f56c0144743854b2a78064dbc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.905838 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.125s user 0.011s sys 0.007s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":23881,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":550,"dirs.run_cpu_time_us":270,"dirs.run_wall_time_us":1040,"lbm_read_time_us":605,"lbm_reads_lt_1ms":27,"lbm_write_time_us":5042,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"mutex_wait_us":6396,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":35147,"threads_started":1,"update_count":27}
I20260809 07:58:48.911358 29211 consensus_queue.cc:1048] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:48.897768 28603 tablet_service.cc:1511] Processing CreateTablet for tablet c3903551936945ccbb1ab823751d3181 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.912776 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3903551936945ccbb1ab823751d3181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.901382 28614 tablet_service.cc:1511] Processing CreateTablet for tablet f96b7f56c0144743854b2a78064dbc0d (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.910454 29405 tablet_bootstrap.cc:492] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:48.913551 29405 ts_tablet_manager.cc:1403] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.006s
I20260809 07:58:48.913782 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f96b7f56c0144743854b2a78064dbc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.914561 28782 tablet_service.cc:1511] Processing CreateTablet for tablet 58bf36691fc149519fb87fe4961b1cf5 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.914693 29263 tablet_bootstrap.cc:654] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.914978 28782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58bf36691fc149519fb87fe4961b1cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.917186 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:48.916813 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.096s user 0.016s sys 0.000s Metrics: {"bytes_written":683,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":120,"dirs.run_wall_time_us":571,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1663,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":17,"update_count":16}
I20260809 07:58:48.917539 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:48.918231 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260809 07:58:48.918292 29405 raft_consensus.cc:359] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.918515 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.342699
I20260809 07:58:48.918536 29405 raft_consensus.cc:385] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.918790 29405 raft_consensus.cc:740] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.919070 29405 consensus_queue.cc:260] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.919644 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"spinlock_wait_cycles":1097984}
I20260809 07:58:48.919972 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"spinlock_wait_cycles":1516288}
I20260809 07:58:48.923686 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":167,"lbm_reads_lt_1ms":4}
I20260809 07:58:48.948155 29474 consensus_queue.cc:1048] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:58:48.960832 28134 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.26.37.1:39223, user_credentials={real_user=slave}} blocked reactor thread for 57226.8us
W20260809 07:58:48.961608 28135 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.26.37.3:38449, user_credentials={real_user=slave}} blocked reactor thread for 48338.4us
I20260809 07:58:48.962594 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.043s user 0.011s sys 0.005s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1663,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":825,"lbm_read_time_us":1347,"lbm_reads_lt_1ms":21,"lbm_write_time_us":1852,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":16}
I20260809 07:58:48.966358 29525 ts_tablet_manager.cc:1929] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.971388 29526 ts_tablet_manager.cc:1929] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:48.974635 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5028 bytes on disk
I20260809 07:58:48.974951 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5028 bytes on disk
I20260809 07:58:48.975203 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5028 bytes on disk
I20260809 07:58:48.975730 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4}
I20260809 07:58:48.976114 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.976336 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:48.967829 29408 tablet_bootstrap.cc:492] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:48.980434 29408 ts_tablet_manager.cc:1403] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.115s user 0.007s sys 0.002s
I20260809 07:58:48.982052 29408 raft_consensus.cc:359] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.982476 29408 raft_consensus.cc:385] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:48.982790 29408 raft_consensus.cc:740] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:48.983099 29408 consensus_queue.cc:260] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:48.984030 29408 ts_tablet_manager.cc:1434] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 07:58:48.984395 29408 tablet_bootstrap.cc:492] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:48.988183 29525 log.cc:1199] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/cda618ed0c4640839f31835d02f906c9
W20260809 07:58:48.980207 28565 consensus_peers.cc:597] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 -> Peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Couldn't send request to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ee878d7fd574bac94938eb05a9612c9. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:48.989632 29526 log.cc:1199] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/ffcb1553e25545ebb774d72295e65fbf
I20260809 07:58:48.990629 29526 ts_tablet_manager.cc:1950] T ffcb1553e25545ebb774d72295e65fbf P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:48.992820 29408 tablet_bootstrap.cc:654] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:48.993458 28617 tablet_service.cc:1511] Processing CreateTablet for tablet 9b66485b6bd343c6bee4269c431d41cb (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.993680 28619 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6305e68f074a63be35264612f6427a (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.991429 28603 tablet_service.cc:1511] Processing CreateTablet for tablet 08bb1e47bc9444338eed65768365c554 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.993947 28617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b66485b6bd343c6bee4269c431d41cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.994302 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6305e68f074a63be35264612f6427a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.992175 28602 tablet_service.cc:1511] Processing CreateTablet for tablet ccc927bb618b4d1b88a976e1ce0d16cd (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.992825 28620 tablet_service.cc:1511] Processing CreateTablet for tablet ed99ebcf81554bd29188729f9bf39c0b (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.991957 28614 tablet_service.cc:1511] Processing CreateTablet for tablet f61b685066df4613b9ad0811be87b144 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.999080 28607 tablet_service.cc:1511] Processing CreateTablet for tablet b47c4be504b14354b7ffb136e0c2b3f7 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.000052 28616 tablet_service.cc:1511] Processing CreateTablet for tablet ffa363422d7142ca9a4a58d5aab41cd7 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.009791 28620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed99ebcf81554bd29188729f9bf39c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.010986 28602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc927bb618b4d1b88a976e1ce0d16cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.011487 28616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffa363422d7142ca9a4a58d5aab41cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.011876 29474 consensus_queue.cc:1048] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260809 07:58:49.015559 28607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47c4be504b14354b7ffb136e0c2b3f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.993060 29525 ts_tablet_manager.cc:1950] T cda618ed0c4640839f31835d02f906c9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:48.995904 28437 tablet_service.cc:1511] Processing CreateTablet for tablet f61b685066df4613b9ad0811be87b144 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.016992 28797 tablet_service.cc:1511] Processing CreateTablet for tablet 08bb1e47bc9444338eed65768365c554 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.017130 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f61b685066df4613b9ad0811be87b144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.017421 28797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bb1e47bc9444338eed65768365c554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.018842 28798 tablet_service.cc:1511] Processing CreateTablet for tablet f61b685066df4613b9ad0811be87b144 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.019325 28798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f61b685066df4613b9ad0811be87b144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:48.998278 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 9b66485b6bd343c6bee4269c431d41cb (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.020684 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b66485b6bd343c6bee4269c431d41cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.021111 28787 tablet_service.cc:1511] Processing CreateTablet for tablet ccc927bb618b4d1b88a976e1ce0d16cd (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.022022 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc927bb618b4d1b88a976e1ce0d16cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.026154 29527 ts_tablet_manager.cc:1950] T 43464de37a144d689c41c6f37d20f6f0 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:48.998734 28435 tablet_service.cc:1511] Processing CreateTablet for tablet ffa363422d7142ca9a4a58d5aab41cd7 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.999096 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6305e68f074a63be35264612f6427a (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:48.999645 28422 tablet_service.cc:1511] Processing CreateTablet for tablet b47c4be504b14354b7ffb136e0c2b3f7 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.027195 28614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f61b685066df4613b9ad0811be87b144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.006038 28432 tablet_service.cc:1511] Processing CreateTablet for tablet ccc927bb618b4d1b88a976e1ce0d16cd (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.007011 28424 tablet_service.cc:1511] Processing CreateTablet for tablet ed99ebcf81554bd29188729f9bf39c0b (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.014633 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 08bb1e47bc9444338eed65768365c554 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.030534 28791 tablet_service.cc:1511] Processing CreateTablet for tablet 9b66485b6bd343c6bee4269c431d41cb (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.032116 28791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b66485b6bd343c6bee4269c431d41cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.035821 28786 tablet_service.cc:1511] Processing CreateTablet for tablet ed99ebcf81554bd29188729f9bf39c0b (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.046630 28793 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6305e68f074a63be35264612f6427a (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.048573 28789 tablet_service.cc:1511] Processing CreateTablet for tablet b47c4be504b14354b7ffb136e0c2b3f7 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.050474 29029 consensus_queue.cc:1048] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260809 07:58:49.052357 28790 tablet_service.cc:1511] Processing CreateTablet for tablet ffa363422d7142ca9a4a58d5aab41cd7 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.052843 28786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed99ebcf81554bd29188729f9bf39c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.058686 28435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffa363422d7142ca9a4a58d5aab41cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.068361 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6305e68f074a63be35264612f6427a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.069736 28790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffa363422d7142ca9a4a58d5aab41cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.069808 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47c4be504b14354b7ffb136e0c2b3f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.070785 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc927bb618b4d1b88a976e1ce0d16cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.070729 29453 consensus_queue.cc:1048] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.071774 28789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b47c4be504b14354b7ffb136e0c2b3f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.072059 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed99ebcf81554bd29188729f9bf39c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.074481 28793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6305e68f074a63be35264612f6427a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.081115 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 43464de37a144d689c41c6f37d20f6f0 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
W20260809 07:58:49.081532 28135 catalog_manager.cc:5017] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): delete failed for tablet e73d79f178374da8b18a76cb1e350939 because the tablet was not found. No further retry: Not found: Tablet not found: e73d79f178374da8b18a76cb1e350939
I20260809 07:58:49.082249 29527 tablet_replica.cc:333] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.082517 29527 raft_consensus.cc:2243] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.082746 29527 raft_consensus.cc:2272] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.085474 28785 tablet_service.cc:1511] Processing CreateTablet for tablet 0e69d7d501fa4bef95a13dc122c6b592 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.085937 28785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e69d7d501fa4bef95a13dc122c6b592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.087726 29527 ts_tablet_manager.cc:1916] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.091092 29001 consensus_queue.cc:1048] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.092748 28438 tablet_service.cc:1511] Processing CreateTablet for tablet 0e69d7d501fa4bef95a13dc122c6b592 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.093308 28438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e69d7d501fa4bef95a13dc122c6b592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.129691 29451 consensus_queue.cc:1048] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.132836 29451 consensus_queue.cc:1048] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.135771 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bb1e47bc9444338eed65768365c554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.136312 29451 consensus_queue.cc:1048] T 160d7600198d40e3a3470352eb70b2c6 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.138595 28603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bb1e47bc9444338eed65768365c554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.143275 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet cda618ed0c4640839f31835d02f906c9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.150959 29527 ts_tablet_manager.cc:1929] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.158236 29527 log.cc:1199] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/cda618ed0c4640839f31835d02f906c9
I20260809 07:58:49.172454 29211 consensus_queue.cc:1048] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.177660 29525 tablet_replica.cc:333] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.177932 29525 raft_consensus.cc:2243] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.178117 29525 raft_consensus.cc:2272] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.178222 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.369s user 0.067s sys 0.064s Metrics: {"bytes_written":683,"cfile_cache_hit":14,"cfile_cache_hit_bytes":27197,"cfile_cache_miss":40,"cfile_cache_miss_bytes":39682,"cfile_init":11,"delete_count":0,"delta_iterators_relevant":10,"dirs.queue_time_us":3891,"dirs.run_cpu_time_us":1454,"dirs.run_wall_time_us":16605,"drs_written":1,"lbm_read_time_us":3291,"lbm_reads_lt_1ms":84,"lbm_write_time_us":13642,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":153069,"reinsert_count":0,"rows_written":88,"thread_start_us":189529,"threads_started":1,"update_count":16}
I20260809 07:58:49.180446 29408 tablet_bootstrap.cc:492] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.180624 29408 ts_tablet_manager.cc:1403] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.196s user 0.004s sys 0.001s
I20260809 07:58:49.181542 29408 raft_consensus.cc:359] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.181806 29408 raft_consensus.cc:385] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.181895 29408 raft_consensus.cc:740] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.182170 29408 consensus_queue.cc:260] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.182752 29408 ts_tablet_manager.cc:1434] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:49.182971 29451 raft_consensus.cc:493] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.183164 29451 raft_consensus.cc:515] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.183846 29451 leader_election.cc:290] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.184787 29525 ts_tablet_manager.cc:1916] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.189657 28820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84fc98b8ab394da1ae55ebd28e2a90d1" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:49.192955 29527 ts_tablet_manager.cc:1950] T cda618ed0c4640839f31835d02f906c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.193585 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet ffcb1553e25545ebb774d72295e65fbf (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.196682 28644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84fc98b8ab394da1ae55ebd28e2a90d1" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.195253 28619 tablet_service.cc:1511] Processing CreateTablet for tablet 0e69d7d501fa4bef95a13dc122c6b592 (DEFAULT_TABLE table=table-4 [id=65f455032f2542f78c40f556537cb794]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260809 07:58:49.199934 28619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e69d7d501fa4bef95a13dc122c6b592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:58:49.215720 29527 tablet_replica.cc:333] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.215838 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet cda618ed0c4640839f31835d02f906c9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.218989 29526 tablet_replica.cc:333] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
W20260809 07:58:49.190373 28382 leader_election.cc:343] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.183044 29408 tablet_bootstrap.cc:492] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.220948 29263 tablet_bootstrap.cc:492] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.221189 29263 ts_tablet_manager.cc:1403] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.372s user 0.019s sys 0.008s
I20260809 07:58:49.221597 29451 consensus_queue.cc:1048] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260809 07:58:49.222049 29263 raft_consensus.cc:359] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.222280 29263 raft_consensus.cc:385] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.222400 29263 raft_consensus.cc:740] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.222754 29263 consensus_queue.cc:260] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.223392 29263 ts_tablet_manager.cc:1434] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:49.223735 29263 tablet_bootstrap.cc:492] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.225834 29263 tablet_bootstrap.cc:654] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
W20260809 07:58:49.226392 28380 leader_election.cc:343] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.226667 28380 leader_election.cc:304] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:49.231258 29527 raft_consensus.cc:2243] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.240181 29527 raft_consensus.cc:2272] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.242931 29527 ts_tablet_manager.cc:1916] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.245412 29211 consensus_queue.cc:1048] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.246136 29211 consensus_queue.cc:1048] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.247320 29014 raft_consensus.cc:2749] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:49.251338 29453 consensus_queue.cc:1048] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.252380 29408 tablet_bootstrap.cc:654] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.254658 29211 consensus_queue.cc:1048] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:58:49.267179 29405 ts_tablet_manager.cc:1434] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.353s user 0.014s sys 0.052s
I20260809 07:58:49.267529 29405 tablet_bootstrap.cc:492] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.267858 29526 raft_consensus.cc:2243] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.268244 29526 raft_consensus.cc:2272] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.274374 29526 ts_tablet_manager.cc:1916] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.287593 29527 ts_tablet_manager.cc:1929] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.287783 29527 log.cc:1199] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/40faafbbf8c646aea67e679524df2cc5
I20260809 07:58:49.288314 29527 ts_tablet_manager.cc:1950] T 40faafbbf8c646aea67e679524df2cc5 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.290251 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 40faafbbf8c646aea67e679524df2cc5 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.301388 29408 tablet_bootstrap.cc:492] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.301560 29408 ts_tablet_manager.cc:1403] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.119s user 0.002s sys 0.011s
I20260809 07:58:49.302448 29408 raft_consensus.cc:359] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.302627 29408 raft_consensus.cc:385] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.302723 29408 raft_consensus.cc:740] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.302959 29408 consensus_queue.cc:260] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.304739 29408 ts_tablet_manager.cc:1434] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 07:58:49.305073 29408 tablet_bootstrap.cc:492] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.306798 29526 ts_tablet_manager.cc:1929] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.307705 29526 log.cc:1199] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/3ee878d7fd574bac94938eb05a9612c9
I20260809 07:58:49.309897 29527 tablet_replica.cc:333] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.317437 29408 tablet_bootstrap.cc:654] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.318199 29263 tablet_bootstrap.cc:492] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.318450 29263 ts_tablet_manager.cc:1403] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.095s user 0.006s sys 0.000s
I20260809 07:58:49.314306 29526 ts_tablet_manager.cc:1950] T 3ee878d7fd574bac94938eb05a9612c9 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.319737 29527 raft_consensus.cc:2243] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.319905 29526 tablet_replica.cc:333] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.320106 29527 raft_consensus.cc:2272] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.320573 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 3ee878d7fd574bac94938eb05a9612c9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.322264 29263 raft_consensus.cc:359] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.322475 29527 ts_tablet_manager.cc:1916] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.322525 29263 raft_consensus.cc:385] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.322716 29263 raft_consensus.cc:740] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.323024 29263 consensus_queue.cc:260] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.325287 29408 tablet_bootstrap.cc:492] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.325513 29408 ts_tablet_manager.cc:1403] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.005s
I20260809 07:58:49.326268 29408 raft_consensus.cc:359] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.326475 29408 raft_consensus.cc:385] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.326568 29408 raft_consensus.cc:740] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.326817 29263 ts_tablet_manager.cc:1434] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260809 07:58:49.326859 29408 consensus_queue.cc:260] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.327399 29408 ts_tablet_manager.cc:1434] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 07:58:49.327720 29408 tablet_bootstrap.cc:492] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.329473 29408 tablet_bootstrap.cc:654] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.333503 29263 tablet_bootstrap.cc:492] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.335187 29263 tablet_bootstrap.cc:654] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.337219 29525 ts_tablet_manager.cc:1929] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.337422 29405 tablet_bootstrap.cc:654] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.338308 29525 log.cc:1199] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/56e4487e4b164b49903280601c1b835d
I20260809 07:58:49.338820 29525 ts_tablet_manager.cc:1950] T 56e4487e4b164b49903280601c1b835d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:49.339810 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 56e4487e4b164b49903280601c1b835d (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.340237 29525 tablet_replica.cc:333] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.340481 29525 raft_consensus.cc:2243] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.340875 29525 raft_consensus.cc:2272] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.343019 29525 ts_tablet_manager.cc:1916] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.343819 29526 raft_consensus.cc:2243] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.345805 29526 raft_consensus.cc:2272] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.346943 29408 tablet_bootstrap.cc:492] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.347142 29408 ts_tablet_manager.cc:1403] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.003s
I20260809 07:58:49.347594 29526 ts_tablet_manager.cc:1916] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.347904 29408 raft_consensus.cc:359] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.348146 29408 raft_consensus.cc:385] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.348294 29408 raft_consensus.cc:740] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.348572 29408 consensus_queue.cc:260] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.349247 29408 ts_tablet_manager.cc:1434] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 07:58:49.349560 29408 tablet_bootstrap.cc:492] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.355281 29526 ts_tablet_manager.cc:1929] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.355370 29408 tablet_bootstrap.cc:654] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.355985 29526 log.cc:1199] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/0dbe33c93dd945feb415cf918022f0d9
I20260809 07:58:49.356629 29526 ts_tablet_manager.cc:1950] T 0dbe33c93dd945feb415cf918022f0d9 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.356774 29453 raft_consensus.cc:493] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.356935 29453 raft_consensus.cc:515] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.357570 29453 leader_election.cc:290] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.357820 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 0dbe33c93dd945feb415cf918022f0d9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.358290 29526 tablet_replica.cc:333] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.358471 29526 raft_consensus.cc:2243] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.358466 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b8280cd463421da5a4813a0df5d61a" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
W20260809 07:58:49.359263 28380 leader_election.cc:343] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.360090 29405 tablet_bootstrap.cc:492] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.360322 29405 ts_tablet_manager.cc:1403] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.093s user 0.000s sys 0.004s
I20260809 07:58:49.358739 29526 raft_consensus.cc:2272] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.361745 29405 raft_consensus.cc:359] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.362075 29405 raft_consensus.cc:385] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.362214 29405 raft_consensus.cc:740] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.362579 29405 consensus_queue.cc:260] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.363180 29405 ts_tablet_manager.cc:1434] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:58:49.363523 29405 tablet_bootstrap.cc:492] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.365204 29577 raft_consensus.cc:493] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.366134 29526 ts_tablet_manager.cc:1916] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.365381 29577 raft_consensus.cc:515] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.371338 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3903551936945ccbb1ab823751d3181" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.371510 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3903551936945ccbb1ab823751d3181" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.372051 29405 tablet_bootstrap.cc:654] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.372184 29577 leader_election.cc:290] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.372295 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b8280cd463421da5a4813a0df5d61a" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:49.372606 29408 tablet_bootstrap.cc:492] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.372828 29408 ts_tablet_manager.cc:1403] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.006s
W20260809 07:58:49.373598 28740 leader_election.cc:343] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:58:49.373952 28740 leader_election.cc:343] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.374248 28740 leader_election.cc:304] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:49.374574 29408 raft_consensus.cc:359] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.374805 28819 raft_consensus.cc:2468] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:49.374914 29211 raft_consensus.cc:2749] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:49.375551 28382 leader_election.cc:304] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:49.376175 29453 raft_consensus.cc:2804] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:49.376418 29453 raft_consensus.cc:493] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:49.376659 29453 raft_consensus.cc:3060] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.378068 29527 ts_tablet_manager.cc:1929] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.378398 29527 log.cc:1199] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/56e4487e4b164b49903280601c1b835d
I20260809 07:58:49.379370 29527 ts_tablet_manager.cc:1950] T 56e4487e4b164b49903280601c1b835d P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.380570 29527 tablet_replica.cc:333] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.380834 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 56e4487e4b164b49903280601c1b835d (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.381807 29453 raft_consensus.cc:515] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.382345 29405 tablet_bootstrap.cc:492] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.383075 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b8280cd463421da5a4813a0df5d61a" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:49.383291 29453 leader_election.cc:290] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.383523 29408 raft_consensus.cc:385] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.383623 29408 raft_consensus.cc:740] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.383414 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b8280cd463421da5a4813a0df5d61a" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
W20260809 07:58:49.383870 28380 leader_election.cc:343] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.383966 28819 raft_consensus.cc:3060] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.383980 29408 consensus_queue.cc:260] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.384184 29405 ts_tablet_manager.cc:1403] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.009s
I20260809 07:58:49.384996 29408 ts_tablet_manager.cc:1434] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.012s user 0.000s sys 0.003s
I20260809 07:58:49.385478 29408 tablet_bootstrap.cc:492] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.385986 29405 raft_consensus.cc:359] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.386271 29405 raft_consensus.cc:385] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.386443 29405 raft_consensus.cc:740] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.386812 29405 consensus_queue.cc:260] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.387611 29408 tablet_bootstrap.cc:654] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.389153 28819 raft_consensus.cc:2468] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:49.389580 29527 raft_consensus.cc:2243] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.389909 29527 raft_consensus.cc:2272] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.390043 28382 leader_election.cc:304] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:49.390417 29453 raft_consensus.cc:2804] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:49.390753 29453 raft_consensus.cc:697] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:49.391047 29453 consensus_queue.cc:237] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.391803 29405 ts_tablet_manager.cc:1434] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 07:58:49.392153 29405 tablet_bootstrap.cc:492] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.395185 29545 raft_consensus.cc:493] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.395349 29545 raft_consensus.cc:515] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.396124 29545 leader_election.cc:290] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.396241 29263 tablet_bootstrap.cc:492] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.396384 29263 ts_tablet_manager.cc:1403] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.000s
I20260809 07:58:49.397589 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a827ea5013564227907d06ef71b0dc58" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:49.398068 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a827ea5013564227907d06ef71b0dc58" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.398314 28462 raft_consensus.cc:2468] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
W20260809 07:58:49.398299 28566 leader_election.cc:343] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.398659 28565 leader_election.cc:304] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:49.399111 29545 raft_consensus.cc:2804] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:49.399293 29545 raft_consensus.cc:493] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:49.399379 29545 raft_consensus.cc:3060] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.399642 29525 ts_tablet_manager.cc:1929] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.401778 29525 log.cc:1199] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/984b64e2ac0441508af6773002acdb97
I20260809 07:58:49.402653 29525 ts_tablet_manager.cc:1950] T 984b64e2ac0441508af6773002acdb97 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:49.402675 29545 raft_consensus.cc:515] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.403502 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a827ea5013564227907d06ef71b0dc58" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:49.403697 28462 raft_consensus.cc:3060] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.403736 29545 leader_election.cc:290] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.403910 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a827ea5013564227907d06ef71b0dc58" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
W20260809 07:58:49.404654 28566 leader_election.cc:343] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.406463 29405 tablet_bootstrap.cc:654] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.407236 29527 ts_tablet_manager.cc:1916] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.407428 28462 raft_consensus.cc:2468] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:49.408070 28565 leader_election.cc:304] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:49.408460 29545 raft_consensus.cc:2804] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:49.409053 29545 raft_consensus.cc:697] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:49.409320 29545 consensus_queue.cc:237] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.409871 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 984b64e2ac0441508af6773002acdb97 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.410630 29525 tablet_replica.cc:333] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.411625 29525 raft_consensus.cc:2243] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.412533 29525 raft_consensus.cc:2272] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.414269 29263 raft_consensus.cc:359] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.414471 29263 raft_consensus.cc:385] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.414597 29263 raft_consensus.cc:740] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.414909 29263 consensus_queue.cc:260] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.416834 29263 ts_tablet_manager.cc:1434] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.020s user 0.002s sys 0.001s
I20260809 07:58:49.417174 29263 tablet_bootstrap.cc:492] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.421204 29408 tablet_bootstrap.cc:492] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.421468 29408 ts_tablet_manager.cc:1403] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.005s
I20260809 07:58:49.422488 29408 raft_consensus.cc:359] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.423739 29408 raft_consensus.cc:385] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.424013 29408 raft_consensus.cc:740] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.416426 29525 ts_tablet_manager.cc:1916] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.425917 29263 tablet_bootstrap.cc:654] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.425549 29408 consensus_queue.cc:260] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.427089 29408 ts_tablet_manager.cc:1434] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260809 07:58:49.431071 29527 ts_tablet_manager.cc:1929] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.434266 29408 tablet_bootstrap.cc:492] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.441264 29453 raft_consensus.cc:493] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.441504 29527 log.cc:1199] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/0dbe33c93dd945feb415cf918022f0d9
I20260809 07:58:49.441504 29453 raft_consensus.cc:515] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.441877 29526 ts_tablet_manager.cc:1929] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.442073 29526 log.cc:1199] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/984b64e2ac0441508af6773002acdb97
I20260809 07:58:49.442103 29527 ts_tablet_manager.cc:1950] T 0dbe33c93dd945feb415cf918022f0d9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.442590 29408 tablet_bootstrap.cc:654] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.443173 29527 tablet_replica.cc:333] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.443400 29527 raft_consensus.cc:2243] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.443632 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 0dbe33c93dd945feb415cf918022f0d9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.443642 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bf36691fc149519fb87fe4961b1cf5" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:49.444506 28382 leader_election.cc:343] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.444767 29453 leader_election.cc:290] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.445173 29527 raft_consensus.cc:2272] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.446307 29526 ts_tablet_manager.cc:1950] T 984b64e2ac0441508af6773002acdb97 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.446409 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bf36691fc149519fb87fe4961b1cf5" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
W20260809 07:58:49.447708 28380 leader_election.cc:343] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.447850 28380 leader_election.cc:304] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:49.448001 29527 ts_tablet_manager.cc:1916] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.448760 29453 raft_consensus.cc:2749] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:49.450433 29526 tablet_replica.cc:333] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.453563 29405 tablet_bootstrap.cc:492] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.458235 29526 raft_consensus.cc:2243] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.458348 28149 catalog_manager.cc:5719] T a827ea5013564227907d06ef71b0dc58 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:49.458490 29526 raft_consensus.cc:2272] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.459384 29405 ts_tablet_manager.cc:1403] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.062s user 0.006s sys 0.009s
I20260809 07:58:49.460913 29405 raft_consensus.cc:359] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.464903 29453 raft_consensus.cc:493] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.465099 29453 raft_consensus.cc:515] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.465917 29453 leader_election.cc:290] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.466287 29527 ts_tablet_manager.cc:1929] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.466524 29527 log.cc:1199] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/5c26d23e8037447c9fdfee1c82f54790
I20260809 07:58:49.468106 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3903551936945ccbb1ab823751d3181" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:49.461143 29405 raft_consensus.cc:385] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.469049 29405 raft_consensus.cc:740] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.469007 28819 raft_consensus.cc:2468] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 0.
I20260809 07:58:49.466778 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3903551936945ccbb1ab823751d3181" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.469357 29405 consensus_queue.cc:260] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.469914 29263 tablet_bootstrap.cc:492] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
W20260809 07:58:49.469852 28380 leader_election.cc:343] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.470209 28382 leader_election.cc:304] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:49.470630 29405 ts_tablet_manager.cc:1434] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260809 07:58:49.470937 29405 tablet_bootstrap.cc:492] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.471015 29453 raft_consensus.cc:2804] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:49.471274 29453 raft_consensus.cc:493] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:49.471378 29453 raft_consensus.cc:3060] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.473030 29405 tablet_bootstrap.cc:654] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.473222 29263 ts_tablet_manager.cc:1403] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.002s
I20260809 07:58:49.474390 29263 raft_consensus.cc:359] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.474787 29263 raft_consensus.cc:385] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.475183 29527 ts_tablet_manager.cc:1950] T 5c26d23e8037447c9fdfee1c82f54790 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.476145 29453 raft_consensus.cc:515] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.477631 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 5c26d23e8037447c9fdfee1c82f54790 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.478091 29527 tablet_replica.cc:333] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.478384 29527 raft_consensus.cc:2243] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.478616 29527 raft_consensus.cc:2272] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.478659 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3903551936945ccbb1ab823751d3181" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:49.478838 29263 raft_consensus.cc:740] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.479200 29453 leader_election.cc:290] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Requested vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
W20260809 07:58:49.479502 28380 leader_election.cc:343] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.479583 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3903551936945ccbb1ab823751d3181" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:49.486330 29263 consensus_queue.cc:260] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.487366 29263 ts_tablet_manager.cc:1434] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260809 07:58:49.487811 29526 ts_tablet_manager.cc:1916] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.490162 28819 raft_consensus.cc:3060] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.493690 29527 ts_tablet_manager.cc:1916] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.494030 29263 tablet_bootstrap.cc:492] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.496765 29263 tablet_bootstrap.cc:654] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.498893 29525 ts_tablet_manager.cc:1929] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.499079 29525 log.cc:1199] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/0dbe33c93dd945feb415cf918022f0d9
I20260809 07:58:49.499632 29525 ts_tablet_manager.cc:1950] T 0dbe33c93dd945feb415cf918022f0d9 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:49.501585 29525 tablet_replica.cc:333] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.501807 29525 raft_consensus.cc:2243] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.501883 28819 raft_consensus.cc:2468] T c3903551936945ccbb1ab823751d3181 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in term 1.
I20260809 07:58:49.502556 29405 tablet_bootstrap.cc:492] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.502664 28382 leader_election.cc:304] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673
I20260809 07:58:49.502772 29405 ts_tablet_manager.cc:1403] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.002s
I20260809 07:58:49.503444 29453 raft_consensus.cc:2804] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:49.504067 29453 raft_consensus.cc:697] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Becoming Leader. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Running, Role: LEADER
I20260809 07:58:49.503983 29405 raft_consensus.cc:359] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.504302 29405 raft_consensus.cc:385] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.504503 29405 raft_consensus.cc:740] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.504549 29453 consensus_queue.cc:237] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.504885 29405 consensus_queue.cc:260] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.505544 29405 ts_tablet_manager.cc:1434] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:58:49.505946 29405 tablet_bootstrap.cc:492] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.506052 29527 ts_tablet_manager.cc:1929] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.506263 29527 log.cc:1199] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/ffcb1553e25545ebb774d72295e65fbf
I20260809 07:58:49.508298 29405 tablet_bootstrap.cc:654] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.508644 29527 ts_tablet_manager.cc:1950] T ffcb1553e25545ebb774d72295e65fbf P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.513523 28153 catalog_manager.cc:5719] T 78b8280cd463421da5a4813a0df5d61a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:49.514776 29314 raft_consensus.cc:493] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.515035 29314 raft_consensus.cc:515] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.515792 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 0dbe33c93dd945feb415cf918022f0d9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.515933 29314 leader_election.cc:290] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.516315 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 984b64e2ac0441508af6773002acdb97 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.517880 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f96b7f56c0144743854b2a78064dbc0d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.519068 29527 tablet_replica.cc:333] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
W20260809 07:58:49.519711 28380 leader_election.cc:343] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.519919 29527 raft_consensus.cc:2243] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.520334 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet ffcb1553e25545ebb774d72295e65fbf (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.520744 29527 raft_consensus.cc:2272] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.521377 29525 raft_consensus.cc:2272] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.522087 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f96b7f56c0144743854b2a78064dbc0d" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:49.522846 28382 leader_election.cc:343] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.522995 29525 ts_tablet_manager.cc:1916] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.523145 28382 leader_election.cc:304] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:49.523433 29527 ts_tablet_manager.cc:1916] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.524226 29211 raft_consensus.cc:493] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.524499 29211 raft_consensus.cc:515] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.524961 29453 raft_consensus.cc:2749] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:49.525204 29211 leader_election.cc:290] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.525839 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bb7ff90e90944e29c8afad95fe45045" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.526176 28642 raft_consensus.cc:2468] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:49.526906 29526 ts_tablet_manager.cc:1929] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.526861 28740 leader_election.cc:304] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:49.527086 29526 log.cc:1199] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/56e4487e4b164b49903280601c1b835d
I20260809 07:58:49.527179 29211 raft_consensus.cc:2804] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:49.527324 29211 raft_consensus.cc:493] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:49.527452 29211 raft_consensus.cc:3060] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.527649 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bb7ff90e90944e29c8afad95fe45045" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.527858 28462 raft_consensus.cc:2468] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:49.530654 29526 ts_tablet_manager.cc:1950] T 56e4487e4b164b49903280601c1b835d P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.530854 29263 tablet_bootstrap.cc:492] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.531024 29263 ts_tablet_manager.cc:1403] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.005s
I20260809 07:58:49.531464 29211 raft_consensus.cc:515] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.532092 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 56e4487e4b164b49903280601c1b835d (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.532094 29263 raft_consensus.cc:359] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.532261 29211 leader_election.cc:290] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.532330 29263 raft_consensus.cc:385] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.532550 29263 raft_consensus.cc:740] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.532825 29263 consensus_queue.cc:260] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.533149 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bb7ff90e90944e29c8afad95fe45045" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:49.533325 28462 raft_consensus.cc:3060] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.533547 29263 ts_tablet_manager.cc:1434] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 07:58:49.534368 29263 tablet_bootstrap.cc:492] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.536355 29263 tablet_bootstrap.cc:654] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.536800 28462 raft_consensus.cc:2468] T 6bb7ff90e90944e29c8afad95fe45045 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:49.537948 28740 leader_election.cc:304] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:49.538297 29211 raft_consensus.cc:2804] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:49.538437 29211 raft_consensus.cc:697] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:49.538820 29211 consensus_queue.cc:237] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.539745 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bb7ff90e90944e29c8afad95fe45045" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:49.540084 28642 raft_consensus.cc:3060] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.544207 28642 raft_consensus.cc:2468] T 6bb7ff90e90944e29c8afad95fe45045 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:49.545137 29408 tablet_bootstrap.cc:492] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.545424 29408 ts_tablet_manager.cc:1403] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.115s user 0.007s sys 0.006s
I20260809 07:58:49.546609 29408 raft_consensus.cc:359] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.547771 29408 raft_consensus.cc:385] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.547960 29408 raft_consensus.cc:740] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.548344 29408 consensus_queue.cc:260] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.549642 29408 ts_tablet_manager.cc:1434] T 9f6305e68f074a63be35264612f6427a P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 07:58:49.550041 29408 tablet_bootstrap.cc:492] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.553956 29526 tablet_replica.cc:333] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.554270 29526 raft_consensus.cc:2243] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.554566 29526 raft_consensus.cc:2272] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.555058 29527 ts_tablet_manager.cc:1929] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.555215 29527 log.cc:1199] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/907d3db7702546e58c219fc805be8f7c
I20260809 07:58:49.555586 29577 raft_consensus.cc:493] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.555853 29577 raft_consensus.cc:515] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.556568 29577 leader_election.cc:290] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.557456 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a827ea5013564227907d06ef71b0dc58" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.557612 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a827ea5013564227907d06ef71b0dc58" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.557929 28462 raft_consensus.cc:2393] T a827ea5013564227907d06ef71b0dc58 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a091b20dbc4ed2b3bf6c960df964c5 in current term 1: Already voted for candidate 24137fd377124538a31e92f3b6df0673 in this term.
I20260809 07:58:49.558401 28740 leader_election.cc:304] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:49.558977 29577 raft_consensus.cc:3060] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.559860 29527 ts_tablet_manager.cc:1950] T 907d3db7702546e58c219fc805be8f7c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.561086 29545 raft_consensus.cc:493] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.561276 29545 raft_consensus.cc:515] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.562094 29545 leader_election.cc:290] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.562567 29577 raft_consensus.cc:2749] T a827ea5013564227907d06ef71b0dc58 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:49.563036 29527 tablet_replica.cc:333] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.563333 29527 raft_consensus.cc:2243] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.563536 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 907d3db7702546e58c219fc805be8f7c (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.563604 29527 raft_consensus.cc:2272] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.564411 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bf36691fc149519fb87fe4961b1cf5" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:49.565948 28566 leader_election.cc:343] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.566460 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bf36691fc149519fb87fe4961b1cf5" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.566735 28462 raft_consensus.cc:2468] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:49.567381 28565 leader_election.cc:304] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:49.568082 29545 raft_consensus.cc:2804] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:49.568213 29545 raft_consensus.cc:493] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:49.568316 29526 ts_tablet_manager.cc:1916] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.568485 29545 raft_consensus.cc:3060] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.569562 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.569903 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.570238 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.570712 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.571002 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.572423 29545 raft_consensus.cc:515] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.573004 29263 tablet_bootstrap.cc:492] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.573207 29263 ts_tablet_manager.cc:1403] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.002s
I20260809 07:58:49.574148 29263 raft_consensus.cc:359] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.574409 29263 raft_consensus.cc:385] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.574678 29263 raft_consensus.cc:740] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.575052 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bf36691fc149519fb87fe4961b1cf5" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:49.575371 29527 ts_tablet_manager.cc:1916] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.575538 29545 leader_election.cc:290] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.575809 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58bf36691fc149519fb87fe4961b1cf5" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:49.576011 28462 raft_consensus.cc:3060] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.577673 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":185,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:49.579751 28462 raft_consensus.cc:2468] T 58bf36691fc149519fb87fe4961b1cf5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:49.579954 28151 catalog_manager.cc:5719] T 6bb7ff90e90944e29c8afad95fe45045 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:49.580516 28565 leader_election.cc:304] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:49.581276 29551 raft_consensus.cc:2804] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:49.581483 29551 raft_consensus.cc:697] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:49.575062 29263 consensus_queue.cc:260] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.581813 29551 consensus_queue.cc:237] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.582052 29263 ts_tablet_manager.cc:1434] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260809 07:58:49.582551 29263 tablet_bootstrap.cc:492] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.582602 29525 ts_tablet_manager.cc:1929] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.582765 29525 log.cc:1199] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/40faafbbf8c646aea67e679524df2cc5
I20260809 07:58:49.583400 29525 ts_tablet_manager.cc:1950] T 40faafbbf8c646aea67e679524df2cc5 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:49.584232 28819 raft_consensus.cc:3060] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.584692 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 40faafbbf8c646aea67e679524df2cc5 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.585040 29263 tablet_bootstrap.cc:654] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.587685 29525 tablet_replica.cc:333] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.587916 29525 raft_consensus.cc:2243] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.588157 29525 raft_consensus.cc:2272] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.589250 28819 raft_consensus.cc:2468] T 58bf36691fc149519fb87fe4961b1cf5 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:49.591425 29526 ts_tablet_manager.cc:1929] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.591588 29526 log.cc:1199] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/40faafbbf8c646aea67e679524df2cc5
I20260809 07:58:49.592149 29526 ts_tablet_manager.cc:1950] T 40faafbbf8c646aea67e679524df2cc5 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.593111 29526 tablet_replica.cc:333] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.593379 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 40faafbbf8c646aea67e679524df2cc5 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.595237 29525 ts_tablet_manager.cc:1916] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.595631 29527 ts_tablet_manager.cc:1929] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.595803 29527 log.cc:1199] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/01bd743cf4eb439eb0e7b86655b630ec
I20260809 07:58:49.596323 29527 ts_tablet_manager.cc:1950] T 01bd743cf4eb439eb0e7b86655b630ec P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.597711 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 01bd743cf4eb439eb0e7b86655b630ec (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.599318 29408 tablet_bootstrap.cc:654] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.601049 29263 tablet_bootstrap.cc:492] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.601289 29263 ts_tablet_manager.cc:1403] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.002s
I20260809 07:58:49.602447 29263 raft_consensus.cc:359] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.602756 29263 raft_consensus.cc:385] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.602916 29263 raft_consensus.cc:740] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.603253 29263 consensus_queue.cc:260] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.603981 29263 ts_tablet_manager.cc:1434] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:58:49.604394 29263 tablet_bootstrap.cc:492] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.606503 29263 tablet_bootstrap.cc:654] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.606827 29527 tablet_replica.cc:333] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.609997 29527 raft_consensus.cc:2243] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.610293 29527 raft_consensus.cc:2272] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.611656 29527 ts_tablet_manager.cc:1916] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.614970 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.615401 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.615873 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.616259 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.617821 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.618161 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.618456 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.618898 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.619350 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.619722 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.620096 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.620476 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.621307 28149 catalog_manager.cc:5719] T c3903551936945ccbb1ab823751d3181 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1). New cstate: current_term: 1 leader_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:49.623003 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.623304 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.623567 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.624511 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.212476
I20260809 07:58:49.624866 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.212476
I20260809 07:58:49.625272 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.212476
I20260809 07:58:49.625193 28153 catalog_manager.cc:5719] T 58bf36691fc149519fb87fe4961b1cf5 P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:49.625872 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.626154 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.631548 29408 tablet_bootstrap.cc:492] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.631737 29408 ts_tablet_manager.cc:1403] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.082s user 0.004s sys 0.000s
I20260809 07:58:49.634356 29526 raft_consensus.cc:2243] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.634626 29526 raft_consensus.cc:2272] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.635231 29527 ts_tablet_manager.cc:1929] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.635361 29527 log.cc:1199] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/3ee878d7fd574bac94938eb05a9612c9
I20260809 07:58:49.635689 29526 ts_tablet_manager.cc:1916] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.635848 29527 ts_tablet_manager.cc:1950] T 3ee878d7fd574bac94938eb05a9612c9 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.637036 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 3ee878d7fd574bac94938eb05a9612c9 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.638193 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.638849 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.639412 29527 tablet_replica.cc:333] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.639671 29527 raft_consensus.cc:2243] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.639878 29527 raft_consensus.cc:2272] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.640053 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.640297 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.007s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":215,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:49.640586 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.642902 29525 ts_tablet_manager.cc:1929] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.643018 29527 ts_tablet_manager.cc:1916] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.643114 29525 log.cc:1199] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/01bd743cf4eb439eb0e7b86655b630ec
I20260809 07:58:49.643636 29525 ts_tablet_manager.cc:1950] T 01bd743cf4eb439eb0e7b86655b630ec P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:49.644858 29525 tablet_replica.cc:333] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.645103 29525 raft_consensus.cc:2243] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.645141 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 01bd743cf4eb439eb0e7b86655b630ec (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.645319 29525 raft_consensus.cc:2272] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.646564 29525 ts_tablet_manager.cc:1916] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.647112 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.004s sys 0.003s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":213,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:49.647869 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.650514 29405 tablet_bootstrap.cc:492] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.633252 29408 raft_consensus.cc:359] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.651108 29405 ts_tablet_manager.cc:1403] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.145s user 0.023s sys 0.054s
I20260809 07:58:49.651379 29408 raft_consensus.cc:385] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.652372 29405 raft_consensus.cc:359] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.652828 29263 tablet_bootstrap.cc:492] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.653012 29405 raft_consensus.cc:385] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.653151 29263 ts_tablet_manager.cc:1403] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.008s
I20260809 07:58:49.653265 29405 raft_consensus.cc:740] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.653812 29405 consensus_queue.cc:260] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.654714 29405 ts_tablet_manager.cc:1434] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 07:58:49.655138 29263 raft_consensus.cc:359] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.655293 29405 tablet_bootstrap.cc:492] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.655495 29263 raft_consensus.cc:385] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.655674 29263 raft_consensus.cc:740] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.656131 29263 consensus_queue.cc:260] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.658393 29263 ts_tablet_manager.cc:1434] T ffa363422d7142ca9a4a58d5aab41cd7 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260809 07:58:49.658831 29263 tablet_bootstrap.cc:492] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.662748 29263 tablet_bootstrap.cc:654] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.663313 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.663815 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.664295 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.666492 29405 tablet_bootstrap.cc:654] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.677935 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.052s user 0.006s sys 0.018s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5401,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":27886,"lbm_read_time_us":1262,"lbm_reads_lt_1ms":27,"lbm_write_time_us":430,"lbm_writes_lt_1ms":9,"mutex_wait_us":5640,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":28380,"threads_started":2,"update_count":2}
I20260809 07:58:49.685971 29527 ts_tablet_manager.cc:1929] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.686124 29527 log.cc:1199] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/984b64e2ac0441508af6773002acdb97
I20260809 07:58:49.686713 29527 ts_tablet_manager.cc:1950] T 984b64e2ac0441508af6773002acdb97 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.687844 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 984b64e2ac0441508af6773002acdb97 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.688352 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":47137}
I20260809 07:58:49.688618 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":40347}
I20260809 07:58:49.688704 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":24038}
I20260809 07:58:49.688983 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":24147}
I20260809 07:58:49.689011 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.001s sys 0.004s Metrics: {"bytes_written":512,"delete_count":0,"lbm_write_time_us":163,"lbm_writes_lt_1ms":4,"mutex_wait_us":41314,"reinsert_count":0,"update_count":4}
I20260809 07:58:49.689823 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.071s user 0.005s sys 0.006s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":917,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":2099,"lbm_read_time_us":205,"lbm_reads_lt_1ms":8,"lbm_write_time_us":254,"lbm_writes_lt_1ms":4,"mutex_wait_us":22091,"reinsert_count":0,"spinlock_wait_cycles":65193344,"thread_start_us":2203,"threads_started":1,"update_count":17}
I20260809 07:58:49.689924 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.007s sys 0.001s Metrics: {"bytes_written":493,"delete_count":0,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"mutex_wait_us":13711,"reinsert_count":0,"update_count":3}
I20260809 07:58:49.690404 29263 tablet_bootstrap.cc:492] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.690361 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.690565 29263 ts_tablet_manager.cc:1403] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.005s
I20260809 07:58:49.690686 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.691051 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.691331 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.691807 29551 raft_consensus.cc:493] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.692270 29263 raft_consensus.cc:359] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.692480 29263 raft_consensus.cc:385] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.692615 29263 raft_consensus.cc:740] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.692762 29408 raft_consensus.cc:740] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.693004 29263 consensus_queue.cc:260] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.693113 29408 consensus_queue.cc:260] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.694253 29551 raft_consensus.cc:515] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.694742 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.694990 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.695128 29263 ts_tablet_manager.cc:1434] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:58:49.695298 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.695533 29408 ts_tablet_manager.cc:1434] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.062s user 0.004s sys 0.000s
I20260809 07:58:49.696923 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b8280cd463421da5a4813a0df5d61a" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:49.697171 28819 raft_consensus.cc:2393] T 78b8280cd463421da5a4813a0df5d61a P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24137fd377124538a31e92f3b6df0673 in current term 1: Already voted for candidate 4ed45b9d1a5d4a1ba3d2dbf1f01c373b in this term.
I20260809 07:58:49.698248 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b8280cd463421da5a4813a0df5d61a" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.698582 29526 ts_tablet_manager.cc:1929] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.700870 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1072,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":437,"lbm_read_time_us":163,"lbm_reads_lt_1ms":8,"lbm_write_time_us":159,"lbm_writes_lt_1ms":4,"reinsert_count":0,"thread_start_us":398,"threads_started":1,"update_count":21}
I20260809 07:58:49.701826 29527 tablet_replica.cc:333] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.702103 29527 raft_consensus.cc:2243] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.702534 29527 raft_consensus.cc:2272] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.702607 29408 tablet_bootstrap.cc:492] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.703203 29551 leader_election.cc:290] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.704782 29527 ts_tablet_manager.cc:1916] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.705109 29408 tablet_bootstrap.cc:654] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.707609 29526 log.cc:1199] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/7ffe5276280a42e3830a3b6551950020
I20260809 07:58:49.708042 28566 leader_election.cc:304] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24137fd377124538a31e92f3b6df0673; no voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5
I20260809 07:58:49.708540 29405 tablet_bootstrap.cc:492] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.708616 29545 raft_consensus.cc:3060] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.708724 29405 ts_tablet_manager.cc:1403] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.054s user 0.002s sys 0.018s
I20260809 07:58:49.709607 29405 raft_consensus.cc:359] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.709808 29405 raft_consensus.cc:385] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.711115 29526 ts_tablet_manager.cc:1950] T 7ffe5276280a42e3830a3b6551950020 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.712323 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 7ffe5276280a42e3830a3b6551950020 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.713084 29551 raft_consensus.cc:493] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.713241 29263 tablet_bootstrap.cc:492] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.713270 29551 raft_consensus.cc:515] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.714362 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f96b7f56c0144743854b2a78064dbc0d" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.714644 28462 raft_consensus.cc:2468] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:49.714704 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f96b7f56c0144743854b2a78064dbc0d" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
I20260809 07:58:49.715049 28819 raft_consensus.cc:2468] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 0.
I20260809 07:58:49.715373 29263 tablet_bootstrap.cc:654] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.715754 29551 leader_election.cc:290] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.715713 29405 raft_consensus.cc:740] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.715744 28565 leader_election.cc:304] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters:
I20260809 07:58:49.716038 29545 raft_consensus.cc:2749] T 78b8280cd463421da5a4813a0df5d61a P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:49.716078 29405 consensus_queue.cc:260] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.716425 29545 raft_consensus.cc:2804] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:49.716689 29545 raft_consensus.cc:493] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:49.716881 29545 raft_consensus.cc:3060] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.720245 29545 raft_consensus.cc:515] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.721217 29545 leader_election.cc:290] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.716754 29405 ts_tablet_manager.cc:1434] T c3903551936945ccbb1ab823751d3181 P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260809 07:58:49.722112 29527 ts_tablet_manager.cc:1929] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.722317 29527 log.cc:1199] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/7ffe5276280a42e3830a3b6551950020
I20260809 07:58:49.722110 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f96b7f56c0144743854b2a78064dbc0d" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5"
I20260809 07:58:49.722623 28819 raft_consensus.cc:3060] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.722795 29526 tablet_replica.cc:333] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.722967 29527 ts_tablet_manager.cc:1950] T 7ffe5276280a42e3830a3b6551950020 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.724152 29527 tablet_replica.cc:333] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.724335 29526 raft_consensus.cc:2243] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.724387 29527 raft_consensus.cc:2243] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.724581 29527 raft_consensus.cc:2272] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.725046 29405 tablet_bootstrap.cc:492] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.726025 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f96b7f56c0144743854b2a78064dbc0d" candidate_uuid: "24137fd377124538a31e92f3b6df0673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:49.726189 28819 raft_consensus.cc:2468] T f96b7f56c0144743854b2a78064dbc0d P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:49.726341 28462 raft_consensus.cc:3060] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.726656 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 7ffe5276280a42e3830a3b6551950020 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.727115 28566 leader_election.cc:304] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:49.727500 29545 raft_consensus.cc:2804] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:49.727627 29545 raft_consensus.cc:697] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Becoming Leader. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Running, Role: LEADER
I20260809 07:58:49.728662 29545 consensus_queue.cc:237] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.729779 29263 tablet_bootstrap.cc:492] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.729985 29263 ts_tablet_manager.cc:1403] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.003s
I20260809 07:58:49.730052 29405 tablet_bootstrap.cc:654] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.730958 29263 raft_consensus.cc:359] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.731171 29263 raft_consensus.cc:385] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.731273 29263 raft_consensus.cc:740] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.731591 29263 consensus_queue.cc:260] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.733044 29263 ts_tablet_manager.cc:1434] T 0e69d7d501fa4bef95a13dc122c6b592 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 07:58:49.733379 29263 tablet_bootstrap.cc:492] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.735118 29263 tablet_bootstrap.cc:654] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.729931 29526 raft_consensus.cc:2272] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.740720 29527 ts_tablet_manager.cc:1916] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.741950 29526 ts_tablet_manager.cc:1916] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.744000 29263 tablet_bootstrap.cc:492] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.744210 29263 ts_tablet_manager.cc:1403] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260809 07:58:49.745411 29263 raft_consensus.cc:359] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.745692 29263 raft_consensus.cc:385] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.745805 29263 raft_consensus.cc:740] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.745999 28462 raft_consensus.cc:2468] T f96b7f56c0144743854b2a78064dbc0d P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24137fd377124538a31e92f3b6df0673 in term 1.
I20260809 07:58:49.746124 29263 consensus_queue.cc:260] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.747550 29263 ts_tablet_manager.cc:1434] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 07:58:49.747814 29263 tablet_bootstrap.cc:492] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.750552 29526 ts_tablet_manager.cc:1929] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.750705 29526 log.cc:1199] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/5c26d23e8037447c9fdfee1c82f54790
I20260809 07:58:49.751325 29526 ts_tablet_manager.cc:1950] T 5c26d23e8037447c9fdfee1c82f54790 P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.752470 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 5c26d23e8037447c9fdfee1c82f54790 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.752849 29526 tablet_replica.cc:333] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.753113 29526 raft_consensus.cc:2243] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.752925 29263 tablet_bootstrap.cc:654] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.753512 29526 raft_consensus.cc:2272] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.754500 29526 ts_tablet_manager.cc:1916] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.757735 29527 ts_tablet_manager.cc:1929] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.761287 28153 catalog_manager.cc:5719] T f96b7f56c0144743854b2a78064dbc0d P 24137fd377124538a31e92f3b6df0673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24137fd377124538a31e92f3b6df0673 (127.26.37.2). New cstate: current_term: 1 leader_uuid: "24137fd377124538a31e92f3b6df0673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260809 07:58:49.766062 29527 log.cc:1199] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/160d7600198d40e3a3470352eb70b2c6
I20260809 07:58:49.766865 29527 ts_tablet_manager.cc:1950] T 160d7600198d40e3a3470352eb70b2c6 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.769160 29405 tablet_bootstrap.cc:492] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.769373 29405 ts_tablet_manager.cc:1403] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.006s
I20260809 07:58:49.770232 29405 raft_consensus.cc:359] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.770471 29405 raft_consensus.cc:385] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.770581 29405 raft_consensus.cc:740] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.770844 29405 consensus_queue.cc:260] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.771447 29405 ts_tablet_manager.cc:1434] T 9b66485b6bd343c6bee4269c431d41cb P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:49.773787 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.774019 29405 tablet_bootstrap.cc:492] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.774083 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.774330 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.774565 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.775104 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.775466 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.775846 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.776207 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.776535 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.776816 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.776836 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.777110 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.777281 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.777326 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.219696
I20260809 07:58:49.777666 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.219696
I20260809 07:58:49.778263 29526 ts_tablet_manager.cc:1929] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.778424 29526 log.cc:1199] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/01bd743cf4eb439eb0e7b86655b630ec
W20260809 07:58:49.778537 28382 consensus_peers.cc:597] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Couldn't send request to peer e2a091b20dbc4ed2b3bf6c960df964c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c26d23e8037447c9fdfee1c82f54790. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:49.778885 29526 ts_tablet_manager.cc:1950] T 01bd743cf4eb439eb0e7b86655b630ec P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.778863 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.779124 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.779359 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.779642 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.779672 29526 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40438 (request call id 123) took 1020 ms. Trace:
I20260809 07:58:49.779868 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.779801 29526 rpcz_store.cc:276] 0809 07:58:48.759361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.760187 (+ 826us) service_pool.cc:224] Handling call
0809 07:58:49.752755 (+992568us) ts_tablet_manager.cc:1151] Deleting tablet 01bd743cf4eb439eb0e7b86655b630ec
0809 07:58:49.752769 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.758358 (+ 5589us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.778182 (+ 19824us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.779613 (+ 1431us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":992397}
I20260809 07:58:49.780987 29526 tablet_replica.cc:333] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.781239 29526 raft_consensus.cc:2243] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.781481 29526 raft_consensus.cc:2272] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.781622 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 01bd743cf4eb439eb0e7b86655b630ec (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.782352 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.782965 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.783380 28137 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.786569 28136 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.005s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":215,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:49.786338 29405 tablet_bootstrap.cc:654] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.790254 29527 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58024 (request call id 128) took 1026 ms. Trace:
I20260809 07:58:49.786592 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":10063,"spinlock_wait_cycles":896}
I20260809 07:58:49.791630 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.004s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1078,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":55,"lbm_read_time_us":161,"lbm_reads_lt_1ms":8,"lbm_write_time_us":135,"lbm_writes_lt_1ms":4,"mutex_wait_us":4439,"reinsert_count":0,"update_count":22}
I20260809 07:58:49.791901 29525 ts_tablet_manager.cc:1929] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.792060 29525 log.cc:1199] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/907d3db7702546e58c219fc805be8f7c
I20260809 07:58:49.792127 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.001s sys 0.004s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:49.792589 29525 ts_tablet_manager.cc:1950] T 907d3db7702546e58c219fc805be8f7c P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
W20260809 07:58:49.793741 28380 consensus_peers.cc:597] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c26d23e8037447c9fdfee1c82f54790. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:49.793110 29527 rpcz_store.cc:276] 0809 07:58:48.763796 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.763881 (+ 85us) service_pool.cc:224] Handling call
0809 07:58:49.724082 (+960201us) ts_tablet_manager.cc:1151] Deleting tablet 160d7600198d40e3a3470352eb70b2c6
0809 07:58:49.724091 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.745300 (+ 21209us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.757664 (+ 12364us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.790195 (+ 32531us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":959940}
I20260809 07:58:49.794611 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 160d7600198d40e3a3470352eb70b2c6 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.786365 29526 ts_tablet_manager.cc:1916] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.797029 29263 tablet_bootstrap.cc:492] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.797236 29263 ts_tablet_manager.cc:1403] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.049s user 0.008s sys 0.013s
I20260809 07:58:49.798090 29263 raft_consensus.cc:359] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.798404 29263 raft_consensus.cc:385] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.798537 29263 raft_consensus.cc:740] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.798890 29263 consensus_queue.cc:260] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.799381 29408 tablet_bootstrap.cc:492] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.799558 29408 ts_tablet_manager.cc:1403] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.097s user 0.002s sys 0.007s
I20260809 07:58:49.801008 29408 raft_consensus.cc:359] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.801312 29408 raft_consensus.cc:385] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.801450 29408 raft_consensus.cc:740] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.801769 29408 consensus_queue.cc:260] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.803148 29408 ts_tablet_manager.cc:1434] T 9b66485b6bd343c6bee4269c431d41cb P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 07:58:49.803532 29408 tablet_bootstrap.cc:492] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.804491 29211 raft_consensus.cc:493] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.805998 29408 tablet_bootstrap.cc:654] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.807360 29211 raft_consensus.cc:515] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.808096 29211 leader_election.cc:290] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.808192 29525 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 122) took 1047 ms. Trace:
I20260809 07:58:49.808331 29525 rpcz_store.cc:276] 0809 07:58:48.760766 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.761095 (+ 329us) service_pool.cc:224] Handling call
0809 07:58:49.644751 (+883656us) ts_tablet_manager.cc:1151] Deleting tablet 907d3db7702546e58c219fc805be8f7c
0809 07:58:49.644760 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.718199 (+ 73439us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.791811 (+ 73612us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.808146 (+ 16335us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":883429}
I20260809 07:58:49.808750 29525 tablet_replica.cc:333] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.809036 29263 ts_tablet_manager.cc:1434] T ed99ebcf81554bd29188729f9bf39c0b P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260809 07:58:49.808998 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc927bb618b4d1b88a976e1ce0d16cd" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.809861 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 907d3db7702546e58c219fc805be8f7c (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.810454 29263 tablet_bootstrap.cc:492] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.810668 29211 raft_consensus.cc:493] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.810750 29527 tablet_replica.cc:333] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.810843 29211 raft_consensus.cc:515] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.810988 29527 raft_consensus.cc:2243] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.811537 29211 leader_election.cc:290] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.811370 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.004s sys 0.008s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5532,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":3365,"dirs.run_cpu_time_us":152,"dirs.run_wall_time_us":707,"lbm_read_time_us":658,"lbm_reads_lt_1ms":27,"lbm_write_time_us":315,"lbm_writes_lt_1ms":9,"mutex_wait_us":8849,"peak_mem_usage":32768,"reinsert_count":0,"update_count":4}
W20260809 07:58:49.812281 28740 leader_election.cc:343] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.812881 29527 raft_consensus.cc:2272] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.813046 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84fc98b8ab394da1ae55ebd28e2a90d1" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.813282 28642 raft_consensus.cc:2468] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:49.813736 28740 leader_election.cc:304] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:49.814222 29211 raft_consensus.cc:2804] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:49.814272 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84fc98b8ab394da1ae55ebd28e2a90d1" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.814356 29211 raft_consensus.cc:493] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:49.814460 29211 raft_consensus.cc:3060] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.814669 29527 ts_tablet_manager.cc:1916] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.817592 29405 tablet_bootstrap.cc:492] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.817498 28462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc927bb618b4d1b88a976e1ce0d16cd" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.817735 29405 ts_tablet_manager.cc:1403] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.001s
I20260809 07:58:49.817761 29211 raft_consensus.cc:515] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.817852 28464 raft_consensus.cc:2468] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
W20260809 07:58:49.818236 28740 leader_election.cc:343] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.818250 29525 raft_consensus.cc:2243] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.818354 28740 leader_election.cc:304] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a091b20dbc4ed2b3bf6c960df964c5; no voters: 24137fd377124538a31e92f3b6df0673, 4ed45b9d1a5d4a1ba3d2dbf1f01c373b
I20260809 07:58:49.818606 29525 raft_consensus.cc:2272] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.818707 29405 raft_consensus.cc:359] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.819012 29577 raft_consensus.cc:2749] T ccc927bb618b4d1b88a976e1ce0d16cd P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:49.819310 29405 raft_consensus.cc:385] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.819447 29405 raft_consensus.cc:740] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.819790 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84fc98b8ab394da1ae55ebd28e2a90d1" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:49.819773 29405 consensus_queue.cc:260] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.820003 28464 raft_consensus.cc:3060] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.820271 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84fc98b8ab394da1ae55ebd28e2a90d1" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:49.820382 29405 ts_tablet_manager.cc:1434] T 9f6305e68f074a63be35264612f6427a P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 07:58:49.820462 28642 raft_consensus.cc:3060] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.823873 28464 raft_consensus.cc:2468] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:49.819597 29525 ts_tablet_manager.cc:1916] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.825469 28740 leader_election.cc:304] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:49.825747 29577 raft_consensus.cc:2804] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:49.826781 28642 raft_consensus.cc:2468] T 84fc98b8ab394da1ae55ebd28e2a90d1 P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:49.826903 29405 tablet_bootstrap.cc:492] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.828293 29211 leader_election.cc:290] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.829061 29405 tablet_bootstrap.cc:654] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.829550 29577 raft_consensus.cc:697] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:49.829828 29577 consensus_queue.cc:237] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.831614 29526 ts_tablet_manager.cc:1929] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.831755 29526 log.cc:1199] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/907d3db7702546e58c219fc805be8f7c
I20260809 07:58:49.832362 29263 tablet_bootstrap.cc:654] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.832420 29526 ts_tablet_manager.cc:1950] T 907d3db7702546e58c219fc805be8f7c P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.833237 29526 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40438 (request call id 122) took 1074 ms. Trace:
I20260809 07:58:49.833420 29526 rpcz_store.cc:276] 0809 07:58:48.759243 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.760396 (+ 1153us) service_pool.cc:224] Handling call
0809 07:58:49.780892 (+1020496us) ts_tablet_manager.cc:1151] Deleting tablet 907d3db7702546e58c219fc805be8f7c
0809 07:58:49.780901 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.799948 (+ 19047us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.831499 (+ 31551us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.833189 (+ 1690us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1020294}
I20260809 07:58:49.833889 29526 tablet_replica.cc:333] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: stopping tablet replica
I20260809 07:58:49.834026 28134 catalog_manager.cc:5050] TS 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): tablet 907d3db7702546e58c219fc805be8f7c (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.834167 29526 raft_consensus.cc:2243] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.834378 29526 raft_consensus.cc:2272] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.837091 29405 tablet_bootstrap.cc:492] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673: No bootstrap required, opened a new log
I20260809 07:58:49.837283 29405 ts_tablet_manager.cc:1403] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.004s
I20260809 07:58:49.838703 29405 raft_consensus.cc:359] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.839010 29526 ts_tablet_manager.cc:1916] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.840194 29525 ts_tablet_manager.cc:1929] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.840479 29525 log.cc:1199] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/5c26d23e8037447c9fdfee1c82f54790
I20260809 07:58:49.841071 29525 ts_tablet_manager.cc:1950] T 5c26d23e8037447c9fdfee1c82f54790 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:49.842038 29525 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 124) took 1080 ms. Trace:
I20260809 07:58:49.842159 29525 rpcz_store.cc:276] 0809 07:58:48.761044 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.761192 (+ 148us) service_pool.cc:224] Handling call
0809 07:58:49.808672 (+1047480us) ts_tablet_manager.cc:1151] Deleting tablet 5c26d23e8037447c9fdfee1c82f54790
0809 07:58:49.808681 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.827485 (+ 18804us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.840114 (+ 12629us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.841965 (+ 1851us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1047215}
I20260809 07:58:49.842595 29525 tablet_replica.cc:333] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.842793 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 5c26d23e8037447c9fdfee1c82f54790 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.843091 29405 raft_consensus.cc:385] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.843508 29405 raft_consensus.cc:740] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24137fd377124538a31e92f3b6df0673, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.843879 29405 consensus_queue.cc:260] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
W20260809 07:58:49.844777 28740 consensus_peers.cc:597] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 -> Peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Couldn't send request to peer 24137fd377124538a31e92f3b6df0673. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:58:49.845257 29527 ts_tablet_manager.cc:1929] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.845467 29527 log.cc:1199] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/00164db27d9243e08c9b667d92c89297
I20260809 07:58:49.846083 29527 ts_tablet_manager.cc:1950] T 00164db27d9243e08c9b667d92c89297 P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.847035 29527 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58024 (request call id 127) took 1083 ms. Trace:
I20260809 07:58:49.846875 28151 catalog_manager.cc:5719] T 84fc98b8ab394da1ae55ebd28e2a90d1 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:49.847183 29527 rpcz_store.cc:276] 0809 07:58:48.763673 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.764100 (+ 427us) service_pool.cc:224] Handling call
0809 07:58:49.810662 (+1046562us) ts_tablet_manager.cc:1151] Deleting tablet 00164db27d9243e08c9b667d92c89297
0809 07:58:49.810672 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.830626 (+ 19954us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.845088 (+ 14462us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.846971 (+ 1883us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1046330}
I20260809 07:58:49.847481 29405 ts_tablet_manager.cc:1434] T ccc927bb618b4d1b88a976e1ce0d16cd P 24137fd377124538a31e92f3b6df0673: Time spent starting tablet: real 0.010s user 0.000s sys 0.003s
I20260809 07:58:49.847607 29527 tablet_replica.cc:333] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: stopping tablet replica
I20260809 07:58:49.847829 29527 raft_consensus.cc:2243] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:58:49.848446 29527 raft_consensus.cc:2272] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.849232 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 00164db27d9243e08c9b667d92c89297 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.850579 29405 tablet_bootstrap.cc:492] T ed99ebcf81554bd29188729f9bf39c0b P 24137fd377124538a31e92f3b6df0673: Bootstrap starting.
I20260809 07:58:49.850982 29527 ts_tablet_manager.cc:1916] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.852623 29405 tablet_bootstrap.cc:654] T ed99ebcf81554bd29188729f9bf39c0b P 24137fd377124538a31e92f3b6df0673: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.852756 29525 raft_consensus.cc:2243] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.853838 29408 tablet_bootstrap.cc:492] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.854023 29408 ts_tablet_manager.cc:1403] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.005s
I20260809 07:58:49.854573 29263 tablet_bootstrap.cc:492] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.854769 29263 ts_tablet_manager.cc:1403] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.045s user 0.000s sys 0.006s
I20260809 07:58:49.854913 29408 raft_consensus.cc:359] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.855121 29408 raft_consensus.cc:385] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.855213 29408 raft_consensus.cc:740] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.855455 29408 consensus_queue.cc:260] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.855576 29263 raft_consensus.cc:359] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.855757 29263 raft_consensus.cc:385] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.855867 29263 raft_consensus.cc:740] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.856099 29408 ts_tablet_manager.cc:1434] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 07:58:49.856232 29263 consensus_queue.cc:260] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.856370 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.856550 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.856714 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.856868 29263 ts_tablet_manager.cc:1434] T 9b66485b6bd343c6bee4269c431d41cb P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:58:49.856925 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.856992 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.857191 29263 tablet_bootstrap.cc:492] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.857254 28140 maintenance_manager.cc:419] P 16fe06ae10064d4b894ac1d45c6ca3f6: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.857291 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260809 07:58:49.857900 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.858271 28138 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.859161 29263 tablet_bootstrap.cc:654] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.859099 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.859385 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.859654 28229 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.860055 29408 tablet_bootstrap.cc:492] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.862013 29408 tablet_bootstrap.cc:654] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.863047 29525 raft_consensus.cc:2272] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.863984 29525 ts_tablet_manager.cc:1916] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:58:49.864663 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.865082 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.865459 28236 maintenance_manager.cc:419] P d2e6b2cd9629422c8bcfa82f3cbad53a: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260809 07:58:49.865870 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.865907 29577 raft_consensus.cc:493] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.866070 29577 raft_consensus.cc:515] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.866134 28234 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260809 07:58:49.866811 29577 leader_election.cc:290] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.868216 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61b685066df4613b9ad0811be87b144" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" is_pre_election: true
I20260809 07:58:49.868480 28464 raft_consensus.cc:2468] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 0.
I20260809 07:58:49.869812 28740 leader_election.cc:304] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:49.869855 29453 raft_consensus.cc:493] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:58:49.870175 29577 raft_consensus.cc:2804] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:58:49.869999 29453 raft_consensus.cc:515] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.870334 29577 raft_consensus.cc:493] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:58:49.870460 29577 raft_consensus.cc:3060] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.870847 29453 leader_election.cc:290] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379), e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449)
I20260809 07:58:49.871732 28819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47c4be504b14354b7ffb136e0c2b3f7" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" is_pre_election: true
W20260809 07:58:49.872396 28382 leader_election.cc:343] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.883527 29408 tablet_bootstrap.cc:492] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.883723 29408 ts_tablet_manager.cc:1403] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.001s
I20260809 07:58:49.883890 29525 ts_tablet_manager.cc:1929] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.884049 29525 log.cc:1199] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-0/wal/wals/7ffe5276280a42e3830a3b6551950020
I20260809 07:58:49.884559 29525 ts_tablet_manager.cc:1950] T 7ffe5276280a42e3830a3b6551950020 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Deleting consensus metadata
I20260809 07:58:49.884759 29408 raft_consensus.cc:359] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.884970 29408 raft_consensus.cc:385] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.885058 29408 raft_consensus.cc:740] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.885283 29527 ts_tablet_manager.cc:1929] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.885324 29408 consensus_queue.cc:260] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.885978 29408 ts_tablet_manager.cc:1434] T 0e69d7d501fa4bef95a13dc122c6b592 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 07:58:49.886361 29408 tablet_bootstrap.cc:492] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.888633 29408 tablet_bootstrap.cc:654] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Neither blocks nor log segments found. Creating new log.
I20260809 07:58:49.895641 29577 raft_consensus.cc:515] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.897467 28464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61b685066df4613b9ad0811be87b144" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b"
I20260809 07:58:49.897711 28464 raft_consensus.cc:3060] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:58:49.897845 28232 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.006s sys 0.001s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":167,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:49.898315 29408 tablet_bootstrap.cc:492] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: No bootstrap required, opened a new log
I20260809 07:58:49.898490 29408 ts_tablet_manager.cc:1403] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.006s
I20260809 07:58:49.898313 28139 maintenance_manager.cc:643] P 16fe06ae10064d4b894ac1d45c6ca3f6: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.007s sys 0.000s Metrics: {"bytes_written":436,"delete_count":0,"lbm_write_time_us":209,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260809 07:58:49.899482 29408 raft_consensus.cc:359] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.899814 29408 raft_consensus.cc:385] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.899943 29408 raft_consensus.cc:740] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.901607 28231 maintenance_manager.cc:643] P d2e6b2cd9629422c8bcfa82f3cbad53a: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1097,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":83,"lbm_read_time_us":209,"lbm_reads_lt_1ms":8,"lbm_write_time_us":216,"lbm_writes_lt_1ms":4,"mutex_wait_us":23734,"reinsert_count":0,"update_count":23}
I20260809 07:58:49.901914 28464 raft_consensus.cc:2468] T f61b685066df4613b9ad0811be87b144 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a091b20dbc4ed2b3bf6c960df964c5 in term 1.
I20260809 07:58:49.902381 29577 leader_election.cc:290] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223), 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379)
I20260809 07:58:49.902447 28740 leader_election.cc:304] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b, e2a091b20dbc4ed2b3bf6c960df964c5; no voters:
I20260809 07:58:49.902583 29527 log.cc:1199] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-2/wal/wals/70a22d70306946e3812d58884686f10c
I20260809 07:58:49.902717 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61b685066df4613b9ad0811be87b144" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.903061 29577 raft_consensus.cc:2804] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:58:49.903180 29527 ts_tablet_manager.cc:1950] T 70a22d70306946e3812d58884686f10c P e2a091b20dbc4ed2b3bf6c960df964c5: Deleting consensus metadata
I20260809 07:58:49.903235 29577 raft_consensus.cc:697] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 1 LEADER]: Becoming Leader. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Running, Role: LEADER
I20260809 07:58:49.903340 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b47c4be504b14354b7ffb136e0c2b3f7" candidate_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673" is_pre_election: true
I20260809 07:58:49.903553 29577 consensus_queue.cc:237] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.904134 29527 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58024 (request call id 126) took 1140 ms. Trace:
I20260809 07:58:49.904281 29527 rpcz_store.cc:276] 0809 07:58:48.763542 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.764332 (+ 790us) service_pool.cc:224] Handling call
0809 07:58:49.847540 (+1083208us) ts_tablet_manager.cc:1151] Deleting tablet 70a22d70306946e3812d58884686f10c
0809 07:58:49.847552 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.874773 (+ 27221us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.882744 (+ 7971us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.904038 (+ 21294us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1082997}
I20260809 07:58:49.904964 29525 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59804 (request call id 125) took 1143 ms. Trace:
I20260809 07:58:49.905176 28135 catalog_manager.cc:5050] TS e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3:38449): tablet 70a22d70306946e3812d58884686f10c (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.905118 29525 rpcz_store.cc:276] 0809 07:58:48.761196 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.761490 (+ 294us) service_pool.cc:224] Handling call
0809 07:58:49.842491 (+1081001us) ts_tablet_manager.cc:1151] Deleting tablet 7ffe5276280a42e3830a3b6551950020
0809 07:58:49.842503 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.878240 (+ 35737us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.883796 (+ 5556us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.904890 (+ 21094us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1080776}
W20260809 07:58:49.905939 28380 leader_election.cc:343] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.906016 28134 catalog_manager.cc:5050] TS 4ed45b9d1a5d4a1ba3d2dbf1f01c373b (127.26.37.1:39223): tablet 7ffe5276280a42e3830a3b6551950020 (table table-3 [id=dcd92b11302d4ac28e8f558b849b0c2d]) successfully deleted
I20260809 07:58:49.906100 28380 leader_election.cc:304] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed45b9d1a5d4a1ba3d2dbf1f01c373b; no voters: 24137fd377124538a31e92f3b6df0673, e2a091b20dbc4ed2b3bf6c960df964c5
W20260809 07:58:49.906455 28740 leader_election.cc:343] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.906574 29453 raft_consensus.cc:2749] T b47c4be504b14354b7ffb136e0c2b3f7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:58:49.907224 29408 consensus_queue.cc:260] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.907462 29525 tablet_replica.cc:333] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: stopping tablet replica
I20260809 07:58:49.907748 29525 raft_consensus.cc:2243] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:58:49.908007 29525 raft_consensus.cc:2272] T 00164db27d9243e08c9b667d92c89297 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:58:49.908787 29263 tablet_bootstrap.cc:492] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5: No bootstrap required, opened a new log
I20260809 07:58:49.908970 29263 ts_tablet_manager.cc:1403] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent bootstrapping tablet: real 0.052s user 0.026s sys 0.022s
I20260809 07:58:49.909283 28647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61b685066df4613b9ad0811be87b144" candidate_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24137fd377124538a31e92f3b6df0673"
I20260809 07:58:49.909660 29526 ts_tablet_manager.cc:1929] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260809 07:58:49.909849 29526 log.cc:1199] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: Deleting WAL directory at /tmp/dist-test-tasktiCCAC/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786261688598168-26772-0/minicluster-data/ts-1/wal/wals/70a22d70306946e3812d58884686f10c
I20260809 07:58:49.909863 29408 ts_tablet_manager.cc:1434] T ffa363422d7142ca9a4a58d5aab41cd7 P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
W20260809 07:58:49.909915 28740 leader_election.cc:343] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24137fd377124538a31e92f3b6df0673 (127.26.37.2:38379): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:58:49.910174 29408 tablet_bootstrap.cc:492] T ccc927bb618b4d1b88a976e1ce0d16cd P 4ed45b9d1a5d4a1ba3d2dbf1f01c373b: Bootstrap starting.
I20260809 07:58:49.911119 29263 raft_consensus.cc:359] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.911372 29263 raft_consensus.cc:385] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:58:49.911473 29263 raft_consensus.cc:740] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a091b20dbc4ed2b3bf6c960df964c5, State: Initialized, Role: FOLLOWER
I20260809 07:58:49.911800 29263 consensus_queue.cc:260] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } }
I20260809 07:58:49.912796 29526 ts_tablet_manager.cc:1950] T 70a22d70306946e3812d58884686f10c P 24137fd377124538a31e92f3b6df0673: Deleting consensus metadata
I20260809 07:58:49.913002 29263 ts_tablet_manager.cc:1434] T 08bb1e47bc9444338eed65768365c554 P e2a091b20dbc4ed2b3bf6c960df964c5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 07:58:49.913374 29263 tablet_bootstrap.cc:492] T b47c4be504b14354b7ffb136e0c2b3f7 P e2a091b20dbc4ed2b3bf6c960df964c5: Bootstrap starting.
I20260809 07:58:49.914304 29526 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40438 (request call id 126) took 1152 ms. Trace:
I20260809 07:58:49.914340 28153 catalog_manager.cc:5719] T f61b685066df4613b9ad0811be87b144 P e2a091b20dbc4ed2b3bf6c960df964c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a091b20dbc4ed2b3bf6c960df964c5 (127.26.37.3). New cstate: current_term: 1 leader_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed45b9d1a5d4a1ba3d2dbf1f01c373b" member_type: VOTER last_known_addr { host: "127.26.37.1" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24137fd377124538a31e92f3b6df0673" member_type: VOTER last_known_addr { host: "127.26.37.2" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2a091b20dbc4ed2b3bf6c960df964c5" member_type: VOTER last_known_addr { host: "127.26.37.3" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260809 07:58:49.914470 29526 rpcz_store.cc:276] 0809 07:58:48.761989 (+ 0us) service_pool.cc:167] Inserting onto call queue
0809 07:58:48.762165 (+ 176us) service_pool.cc:224] Handling call
0809 07:58:49.833800 (+1071635us) ts_tablet_manager.cc:1151] Deleting tablet 70a22d70306946e3812d58884686f10c
0809 07:58:49.833813 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0809 07:58:49.854122 (+ 20309us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.882151 (+ 28029us) tablet_metadata.cc:665] Metadata flushed
0809 07:58:49.914230 (+ 32079us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us"